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FOREWORD 



This monograph focuses on the findings and results of 
an intensive study of areawide planning for the imple- 
mentation of Title II A, B, and C (formerly Title I) of 

the Comprehensive Employment and Training Act (CETA) , 
as performed by prime sponsors to meet the needs of 
their communities. 

The most general finding of the report is that careful 
planning does have the potential for helping to improve 
program performance, but the report points out very 
clearly that many other aspects of prime sponsorship 
management need to be handled well in order to achieve 
programmatic goals. In Section IV, the report develops 
three models of planning that were observed in the 
field work studies, as well as a group of "exemplary 
approaches to critical elements of planning and 
management." Also in Section IV, a number of 
productive approaches to seven specific planning and 
management elements are described because of their 
favorable programmatic impact and their potential 
utility in additional prime sponsorships. 

Observations and specific recommendations for 
improving prime sponsor activities are included 
for review and implementation. 



HOWARD ROSEN 
Director 

Office of Research 
and Development 



PREFATORY NOTE 



This is the last report in a series that focuses on planning for Title I 
of the Comprehensive Employment and Training Act of 1973 (Title II ABC of CETA 
as reauthorized in 1978) at the prime sponsorship level. The central 
objectives of the research have been to 1) describe Title I planning systems 
that have emerged; 2) relate the features of planning systems to contextual 
factors; and 3) explore the links between Title I planning systems and program 
performance. 

This project has been supported by a grant from the Office of Research and 
Development of the Employment and Training Administration of the U.S. Department 
of Labor (No. 21-39-75-10) and by resources of the Mershon Center of Ohio State 
University. Additional support of various kinds has come from the Department 
of Political Science at Ohio State University, the Instruction and Research 
Computer Center at Ohio State, the Department of Government at Smith College 
(the home institution of Professor Donald Baumer, a project associate), and the 
Eagleton Institute of Politics at Rutgers University (the home institution of 
Professor Carl Van Horn, a project associate). 

The Director, Associate Director, and Project Associates did field work 
in 12 prime sponsorships throughout the United States that were chosen to be 
illustrative of areas in which Title I planning is taken seriously and to be 
representative of different kinds of areas in terms of regional location, size 
of CETA program, type of prime sponsorship, and labor market conditions. The 
field work was done from March through August 1978. Detailed reports on six of 
the sites are included in a progress report dated June 30, 1978; similar reports 
on the other six sites are included in a progress report dated September 30, 1978. 

The authorship of this report is genuinely collegial. Primary credit for 
the analysis in Section II belongs to Michael ! Loughlin, Marcia Slotnick, 
William Strangfeld, and Richard Wright. Primary credit for the analysis in 
Section III belongs to Grace Franklin and John Wichita. Franklin and Strangfeld 
are primarily responsible for the coordination between the two sections. 
Primary credit for the analysis in Section IV belongs to Donald Baumer and 
Carl Van Horn. The Director, Randall Ripley, takes final responsibility for 
the contents of the entire report. 

We are grateful to a large number of individuals in the prime sponsorships 
we visited and also to a large number of ETA employees for their splendid coopera- 
tion. Many participated in long interviews. Others provided other kinds of 
essential data. Without such cooperation this research would have been 
impossible. We are also grateful to Professor Aage Clausen of Ohio State for 
his advice on the specific use of multiple regression techniques in Section III. 
Finally, we are very appreciative of the continuing support and work of 
Richard McAllister, our project officer in the Office of Research and Development 
and, in our opinion, the model of what a good project officer should be. 
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SUMMARY 



The report focuses on planning for Title I (now Title II A, B, and C) 
of CETA at the prime sponsorship level. Intensive field work was done in 12 
prime sponsorships selected, in part, because they were reputed to take planning 
seriously. Results are compared on some questions with national averages 
and with observations in 30 other prime sponsorships in which intensive field 
work was done for previous studies. 

The most general finding is that careful planning by prime sponsorship 
staff does, indeed, have the potential for helping improve program perfor- 
mance. At the same time, it is quite clear that planning is only one of a 
number of general aspects of prime sponsorship management that needs to be 
handled well in order to enhance the chances of achieving programmatic goals. 
Fortunately, these elements of management are highly manipulable at the local 
level. Also fortunately, non -manipulable elements of local context such as 
economic conditions do not create severe restraints on what can be achieved. 
In addition, there is solid evidence that prime sponsorships can target 
their resources on the most disadvantaged part of the eligible population as 
measured by gross demographic characteristics without diminishing their 
potential for good program performance. 

MODELS OF PLANNING 

Planning is treated as one potentially important structured intervention 
between contextual factors in a prime sponsorship and the results of programs. 
There are both more "technical" and more "political" aspects of planning. 
Both aspects are important and they interact with each other in influencing 
program results. 

Three models of planning were developed on the basis of work in 12 prime 
sponsorships. A fourth model (no consistent attention to planning) is also 
implicit. The three empirically developed models in which planning is taken 
seriously (even though there are severe problems in one case) can be summarized 
briefly: 

1. A crisis management model characterized by: 

a. Unstable relations among actors 

b. Unmanaged conflict 

c. Lack of routine 

d. Malfunctioning feedback system. 

2. An operations management model characterized by: 

a. Stable relations among actors 

b. Well-managed conflict 

c. Many routine procedures 

d. Feedback mechanism in place and utilized. 

xi 



3 A future oriented model characterized by* 

a. The four characteristics of the operations management model 

b. Deliberate attention to long-range decisions* 

Both Operations Management and Future Oriented models are superior to No 
Consistent Planning and to Crisis Management models. But there is good 
empirical evidence that under different conditions either Operations Management 
or Future Oriented planning will be relatively the most productive. There is 
not one "correct" way to plan in all situations, 

THE EFFECTS OF CONTEXT ON PLANNING SYSTEMS 

General contextual factors (nature of local economy, nature of local 
population, program size, and organizational nature) are not determinative of 
what kind of planning systems emerge in individual prime sponsorships. Locali- 
ties can design the kind of planning system they choose for programmatic reasons 
and are not forced into a specific planning mode by these external factors. 

The general management context in a prime sponsorship is, however, related 
to the kind of planning system that seems most likely to emerge. This is 
probably because certain styles of planning fit best with certain styles of 
management; associational patterns were evident when future-oriented and 
operations management sites were compared (only one site has a crisis management 
system and we did not, of course, attempt to generalize from it). 

In prime sponsorships with future-oriented planning systems, the associated 
management characteristics included a pattern of staff -dominated decision-making 
(with only limited influence sharing with other actors) , a stable staff of high 
quality, stronger than average support for staff from political officials, 
thorough monitoring, and the active involvement of the private business sector. 

In prime sponsorships with operations management planning systems, there 
was more likely to be less staff domination of decision-making (usually involving 
significant sharing of influence with service deliverers) , more turnover in key 
staff, low involvement of business, and less well-developed monitoring systems. 

EXPLAINING PROGRAM PERFORMANCE 

In discussing program performance we used multiple measures. Some were 
observational and judgmental, based on qualitative data and impressions. We 
also used five quantitative measures, although we noted the shortcomings both 
of the data and of the measures themselves: 

1. Overall placement rate (the number of participants entering employ- 
ment divided by the number enrolled) . 

2. Placement efficiency (the number of participants indirectly placed 
divided by the number enrolled). 

3. Nonpositive termination rate (the number of nonpositive terminations 
divided by the number of all terminations) . 

4. Cost per enrollee (total accrued expenditures divided by the number 
of enrollees) . 



5. Cost per placement (total accrued expenditures divided by the 
number of persons placed) * 

Management characteristics and planning both have stronger influence on 
performance than any contextual variables. We found no empirical support for 
the conventional wisdom that suggests that economic conditions and demographic 
characteristics of participants determine or at least highly constrain 
performance. 

The factors most strongly associated with good performance were quality of 
staff, the nature and extent of business involvement , and the quality of 
monitoring- Other factors were also associated, but more moderately so. 

EXEMPLARY APPROACHES TO CRITICAL ELEMENTS OF PLANNING AND MANAGEMENT 

Because a variety of management and planning factors were found to affect 
program performance we spelled out in some detail specific instances of 
approaches to some key elements of planning and management we had observed to 
be working well. We found a number of productive approaches to seven specific 
planning and management elements we thought should be described because of 
their favorable programmatic impact and their potential utility in additional 
prime sponsorships. The elements on which we focused were: 

1. Manpower planning councils 

2. Monitoring and evaluation 

3. Universe of need and target group identification 

4. Intake and assessment strategies 

5. Labor market analysis 

6. System design (service delivery selection, delivery systems, 
performance contracting) 

7. Participant placement strategies 



I . INTRODUCTION 



The Comprehensive Employment and Training Act of 1973 (CETA) represents 
a compromise form of special revenue- sharing or block grant enacted following 

5 years of struggle between Congress and the Administration. (For excellent 
background on that struggle and on the various manpower programs preceding 
CETA see Advisory Commission on Intergovernmental Relations, 1977; Davidson, 
1972; Levitan and Taggart, 1974; and Levitan and Zickler, 1974.) As a form 
of revenue-sharing, it embodies a substantial restructuring of most Federal 
manpower programs, mandating decentralization and decategorization. 
Responsibility for the transition to and operation of decentralized and 
decategorized programs is lodged firmly with local officials (and, to a lesser 
extent, with State officials) by the statute. 

The statute and regulations are long and complicated. Table 1 summarizes 
the major provisions of the 1973 act as amended in 1974 and 1977. (The 1978 
amendments are not included since our research dealt with the program as it 
existed under the provisions contained in Table 1. ) 

A number of assessments of CETA operations have appeared, some focused on 
specific aspects and some quite broad in character (for several broad assessments 
see Mirengoff and Rindler, 1978; Snedeker and Snedeker, 1978; and Van Horn, 1979). 
However, despite the presumed centrality of local planning in making CETA perform 
better than the pre-CETA categorical programs, little systematic attention has 
been given to planning. This report seeks to fill that gap. 

A. CONCEPTUAL FRAMEWORK 

The Comprehensive Employment and Training Act embodies the belief that local 
prime sponsors know best how and when to respond to what specific local conditions 
in order to achieve the general goals of the program. Nationally, there has been 
great variation among prime sponsorships in terms of types of conditions faced, 
the types of programmatic responses generated, and the results of the responses. 

In an earlier study (Ripley and associates, 1978) we explored a number of 
management decision areas at the local level to ascertain how they were affected 
by local conditions and, critically, how they in turn affected local program 
performance. Our central research question in that study was: under what 
conditions do what management decision choices seem most likely to enhance 
desired program performance? 

In the present study we focus on planning as a special aspect of management. 
We are interested in describing the types of planning systems that have emerged, 
in relating the features of those systems to contextual factors, and in exploring 
the links between planning systems and program performance. 

Despite considerable literature, buttressed by rhetoric from both national 
and local sources, that asserts that planning is central to good performance 
in employment and training programs there is little empirical work that assesses 
the nature of planning that occurs and the impact of that planning on programs. 
This paucity of empirical research on planning and its consequences is especially 
ironic in the case of CETA since one of the explicit goals of CETA is "comprehensive 
programs" and one of its implicit goals is "comprehensive planning." The Act 
and the Department of Labor encourage comprehensiveness in both programs and 



Table 1: SUMMARY OF THE COMPREHENSIVE 
EMPLOYMENT AND TRAINING ACT (CETA) OF 1973 

The Comprehensive Employment and Training Act of 1973 (PL 93-203) , 
as amended by the Emergency Jobs and Unemployment Assistance Act of 1974 
(PL 93-567) , by the Emergency Jobs Programs Extension Act of 1976 (PL 94-444) , 
by the Comprehensive Employment and Training Act Amendments of 1977 
(PL 95-44), and by the Youth Employment and Demonstration Projects Act of 
1977 (PL 95-93) has eight titles: 

Title I authorizes comprehensive manpower services for the unemployed, 
underemployed, and economically disadvantaged. Programs are administered 
by prime sponsors, which are cities and counties of 100,000 or more, and 
consortia. The State government is prime sponsor for the balance of state. 
Funds are allocated according to each area f s prior year f s apportionment, 
number of unemployed, and adults in low-income families. Prime sponsors 
must submit an acceptable plan to the Secretary of Labor, prepared in con- 
sultation with local advisory councils. A State manpower services council 
reviews local plans and arranges for the cooperation of State agencies. 

Title II provides funds to prime sponsors and Indian reservations to 
hire the unemployed in areas of substantial unemployment for public service 
jobs. Funds are allotted on the basis of the number of unemployed. 

Title III provides for nationally administered programs for Indians, 
migrant and seasonal farmworkers, , youth, and other groups that are in 
particular need of such services. This title also gives the Secretary of 
Labor responsibility for research, evaluation, experimental and demonstra- 
tion projects, labor market information, and job banks. 

Title IV authorizes the Department of Labor to operate the Job Corps, 
residential training centers for disadvantage^ young men and women. 

Title V establishes a National Commission for Manpower Policy to identify 
goals, evaluate manpower development programs, and make recommendations 
to the President and to Congress. (The Emergency Jobs Programs Extension 
Act of 1976 establishes a separate National Commission on Employment and 
Unemployment Statistics.) 

Title VI authorizes public service jobs for the unemployed. Funds are 
allocated to prime sponsors and Indian tribes, based on the number of un- 
employed, the unemployed in excess of a 4.5-percent rate, and the unemployed 
in areas of substantial unemployment. Under 1976 amendments, funds for the 
expanded Title VI program are in new short-duration projects and most new 
participants must be long-term, low-income unemployed or welfare recipients. 

Title VII contains provisions applicable to all programs such as 
prohibitions against discrimination and political activity. 

Title VIII establishes a Young Adult Conservation Corps to carry out 
projects on public lands. 

Source: Committee on Evaluation of Employment and Training programs, 
CETA: Assessment and Recommendations (Washington: National 
Academy of Sciences, 1978), p. 4. 



planning by promoting consortia as one way of putting labor markets together. 
The Department of Labor also provides written materials to prime sponsors on 
"how to" plan* But the manpower literature offers only meager support for 
any prescriptions or on the effects of different ways of planning. Even more 
surprising, the literature does not even offer very rich descriptions of 
various methods of planning. Any specific analysis often assumes there is only 
one right way to plan. 

The empirical literature on the first few years of CETA experience at 
the prime sponsorship level makes it clear that different planning systems 
do exist, that planning is difficult to generate at the local level , and that 
the consequences of different planning systems are not well known. 

Our conception of planning is very broad. We harbor the simple notion 
that any planning system is set in a broad local context that directly affects 
both the nature of that system and program results. We also assume that, 
within broad limits related to the context, the nature of the planning system 
has the potential for affecting program results. 

In a generic sense we regard planning as one potentially important 
structured intervention between the contextual factors in a prime sponsorship 
and the results of programs. Theoretically, "good" planning should result 
in improved program performance because it will help in the development of the 
most productive programmatic responses to contextual factors and /or it may 
help in altering those portions of the context that are susceptible to 
deliberate manipulation in the short run. 

There are both more "technical" and more "political" aspects of planning. 
Some parts of planning are focused primarily on the technicalities of labor 
market information and programmatic response whereas other parts more broadly 
assess the needs of the community and the appropriate programmatic responses 
to those needs in broad, nontechnical, political terms. Our previous work 
on CETA leads us to believe that both the technical and political aspects 
of planning are important and that they interact with each other in 
influencing program results (see Ripley and associates, 1977, 1978). 

The present research is set in the context of two previous projects we 
have completed. Between mid-1974 and mid-1976 we examined the implementation 
of CETA in 17 prime sponsorships in one large State (Ohio). We produced an 
elaborate map of those 17 experiences and the commonalities we observed in 
them. We also sought to explain those commonalities. (For the final 
report from that project, see Ripley and associates, 1977.) 

In 1976 and 1977 we concentrated on a number of broadly defined aspects 
of prime sponsorship management and sought to link management decisions in 
those areas to program goal achievement in 15 prime sponsorships spread 
throughout the country, comparing the experience of the 15 with the 17 Ohio 
cases wherever possible. (For the final report from that project, see Ripley 
and associates, 1978.) 

The earlier research provides us both with a number of empirically 
supported findings that are relevant to our present work and with two comparison 
groups of prime sponsorships against which our findings on planning can be 
"tested," at least intuitively. (There were only two cases of overlap in 
the three clusters of prime sponsorships used for these studies: two of the 
planning sites had also been visited in the management study.) 



B. DATA BASE 

We systematically collected data on a number of different elements of 
the local context for planning, the nature of local planning, and program 
results in the 12 prime sponsorships in which we worked* The following list 
summarizes those elements: 

I. LOCAL CONTEXT FOR PLANNING 

A. HISTORY OF TITLE I PROGRAMS 

1. CETA budget resources 

2. Allocation of budget at local level 

3. Distribution of participants among program components 

4. Past program performance of prime sponsorship 

5. Characteristics of persons enrolled and placed 

6. Service deliverers 

7. Degree, nature of change in the employment and training 
delivery system 

8. Degree, nature of conflicts in the employment and training 
delivery system 

B. LOCAL ECONOMIC CONDITIONS (LOCAL LABOR MARKET) 

1. Characteristics of local labor force 

2. Unemployment rate 

3. Current employment opportunities 

4. Projected employment opportunities 

5. Nature of unemployment 

C. DEMOGRAPHIC CONDITIONS IN PRIME SPONSORSHIP 

1. Characteristics of general population 

2. Characteristics of those most in need of manpower services 

D. CHARACTERISTICS OF CETA STAFF 

1. Administrative location of CETA units 

2. Staff organization 

3. Attitudes /commitments of staff 

4. Professional qualifications of staff 

E. CHARACTERISTICS OF PRIME SPONSORSHIP 

1. Jurisdictional type 

2, Local decision-making patterns 

F. HISTORY OF TITLE I PLANNING 

1. Staff attention to planning over time 

2. Staff cooperation with other planning agencies 

3. Change in planning over time 

II. NATURE OF LOCAL PLANNING 

A. STAFF CONCEPTION OF PLANNING 

B. LOCAL GOALS OF THE CETA PROGRAM 



C. TARGET GROUPS TO BE SERVED 

D. BARRIERS TO EMPLOYMENT 

E. IDENTIFYING JOB OPENINGS 

F. PRESCRIBING A MIX OF SERVICES 

G. SERVICE DELIVERERS 1 SELECTION 

H. MONITORING AND EVALUATION PROCESSES 

I. USE OF MONITORING AND EVALUATION 

J. PUBLIC PLANNING PROCESS 

K. STAFF TIES WITH OTHER PLANNING UNITS 

L. SEPARATION/INTEGRATION OF PLANNING AND OPERATIONS 

M. SEPARATION/INTEGRATION OF TITLE I PLANNING WITH OTHER CETA 
PLANNING 

N. USE OF SOCIOECONOMIC DATA IN PLANNING 
III. PROGRAM RESULTS 

A. LOCAL GOAL ACHIEVEMENT 

B. OCCUPATIONAL TRAINING AREAS 

C. PARTICIPANT CHARACTERISTICS 

1. All enrollees 

2. All persons placed 

3. Local significant segment groups 

D. PROGRAM MIX 

E. PROGRAMMATIC INTEGRATION 

F. STANDARD PERFORMANCE MEASURES (PLACEMENT RATES AND COST) 

G. QUALITY OF PLACEMENTS 
H. NON-CETA LINKAGES 

Data of four kinds were collected: 1) quantitative data such as that 
contained in prime sponsors' quarterly reports; 2) a wide variety of information 
contained in documents such as local plans, modifications, and public information 
releases; 3) perceptions, observations, and opinions of individuals involved in 
CETA systems at the local level; and 4) direct observation by the researchers 
of specific processes such as advisory council meetings. 

We got the needed data in the first category from the national office of 
the Employment and Training Administration. We collected the data in the 
second category from the files of the 12 prime sponsorships. 



Data in the third category were collected in over 400 interviews in 
the 12 prime sponsorships with individuals of eight kinds i 1) the director 
of the CETA staff and the chief planner; 2) other professional CETA staff 
members; 3) Employment Service officials who interact with CETA in planning; 

4) other local planners who interact with CETA in planning (for example , 
planners for the chamber of commerce or a regional planning commission) ; 

5) the most informed and active members of the advisory council; 6) the 
Federal representative for the prime sponsorship; 7) service deliverers; 
and 8) political officials. 

Data in the fourth category came from observation of about 50 different 
meetings (staff meetings, planning staff meetings, advisory council meetings) 
in the 12 prime sponsorships. 

C. SITE SELECTION AND DESCRIPTION 

In choosing 12 sites for field work we wanted prime sponsorships that 
were at least reputed to take Title I planning seriously. We used three 
primary sources for arriving at an initial list of "nominees." 

First, we contacted a number of people knowledgeable about planning in CETA 
to describe our project and ask for their nominations. These individuals include 
staff members of the National League of Cities-U.S. Conference of Mayors; the 
National Association of Counties; the National Manpower Commission; regional 
ETA offices; Institutional Grant universities; other research organizations 
that have studied aspects of CETA (the National Academy of Sciences, the Manpower 
Development Corporation, and IMPACT); and individual prime sponsors. 

Second, we went through all of the published and unpublished literature 
on CETA in our files to look for possible sites. 

Third, we reviewed our own files on 32 prime sponsorships in which we had dc 
intensive field work. 

From these "nominations" we chose a purposive sample that would maximize 
variation in 1) the type of prime sponsorship; 2) the general economic 
position of the prime sponsorship; 3) size of the prime sponsorship in terms of 
dollar allocations; and 4) geographic spread. 

Type of Prime Sponsorship 

For purposes of both planning and linking planning to operations^ the 
major difference is between multiple jurisdiction prime sponsorships (consortia) 
and single jurisdiction prime sponsorships (cities and counties). Our sample 
of 12 included seven consortia and five single jurisdiction prime sponsorships. 

General Economic Situation 

We looked at two general aspects of economic situation: economic health 
as measured by unemployment and position in a labor market. Seven of our 
prime sponsorships had unemployment higher than the national average, with 
two of those seven being well above average. Five of our sites had unemployment 
lower than the national average, with two of those five being well below 
average. Our sites were about evenly split between those that dominate a 
labor market and those that are only part of a labor market. 



Size of Program 

We used the dollar amount of Fiscal Year 1978 Title I allocations (before 
consortium incentives) as an index of program size. Three of our prime sponsor- 
ships fell between $7.8 million and $13.8 million. Three fell between $3.7 
million and $5.6 million. Three fell between $2.8 million and $3.0 million. 
And three fell between $1.2 million and $2.2 million. We excluded prime sponsor- 
ships with an allocation of less than $1 million from consideration as sites. 

Geographical dispersion 

We had at least one site in 9 of the 10 Federal regions and had a second 
site in three of the larger regions. The only region in which no site was 
selected was Region V. This was the case for two reasons: First, none of 
our contacts nominated a prime sponsorship in this region for study; second, 
we knew 18 prime sponsorships in the region well already and could use them for 
purposes of comparison with our 12 sites on some points. 

Sites Chosen 

Our first twelve preferences all agreed to participate in the project. In 
each case we contacted the staff director and explained the nature of the 
project and the nature of the field work and solicited his or her cooperation. 
The field work was conducted in the following 12 prime sponsorships: 

Penobscot Consortium, Maine 

Syracuse, New York 

Bergen County, New Jersey 

Baltimore Consortium, Maryland 

Atlanta, Georgia 

Heartland Consortium, Florida 

Gulf Coast Consortium, Texas 

Albuquerque Consortium, New Mexico 

Omaha Consortium, Nebraska 

Denver , Colorado 

San Francisco, California 

King-Snohomish Consortium, Washington 

Table 2 contains some summary information on these prime sponsorships. 
Appendix A contains a short description of some of the most important aspects 
of what we observed in each of the 12 prime sponsorships at the time we visited 
(indicated on Table 2). It should be underscored that some features in these 
12 sites were no doubt different before we took our snapshot and have 
probably changed since. 

D. ORGANIZATION OF THE REPORT 

Three major analytical sections follow. The first focuses on planning 
systems, including the effects of local context on the nature of planning. 
The second focuses on explaining program performance. The third discusses 
exemplary approaches to critical elements of planning and management. A short 
concluding section offers some summary comments. 
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II. PLANNING SYSTEMS 



The purpose of section II is three~fold. First, we will briefly discuss 
the nature of planning in general. Second, we will develop three models of 
planning systems. Third, we will elaborate on the empirical referents for those 
three models. Fourth, we will discuss the effect of context and management 
decisions on planning. 

A. TEE NATURE OF PLANNING 

We view planning activities as a specific subset of general management 
activities. Thus planning decisions are a subset of general management 
decisions. The planning decision-making process is a special instance of 
general management decision-making. Both management and planning activities, 
decisions, and decision-making processes are set within a general context. 
The following figure suggests the inter-relationships between general context, 
management decisions, and planning decisions- The following points should be 
especially noted: (1) management decisions include planning decisions, (2) 
planning decisions are separable from management decisions for purposes of 



^MANAGEMENT 
DECISIONS 

PLANNING 
DECISIONS 




analysis, and (3) from the perspective of planning decisions, context represented 
by the shaded area in the figure consists of both general context and management 
decisions. 

Planning decisions cover five major areas: 

identification and choice of local goals; 

identification and choice of participant target groups to be 
served; 

identification of target occupations for training and placement; 

identification and choice of program mix; and 



identification and selection of service deliverers who are to 
deliver the training and employment services. 



These decision areas are defined broadly in each case, so that the 
service deliverer decision area, for example, includes not only such decisions 
as which deliverers to fund, but also of whether to use an RFP and whether to 
contract or operate services in-house. 

We have limited what we mean by "planning 11 to that which is included in 
these five decision areas knowing that alternative definitions of planning are 
possible. Others, for example, have separated planning into strategic and 
operational components. The merit of the five decisions we have selected is 
that they exactly cover all of the basic choices that must be made to 
accomplish the goal any prime sponsor must address, namely (in the words of the 
statute) that of providing "job training and employment opportunities for 
economically disadvantaged, unemployed, or underemployed persons which will 
result in an increase in their earned income..." By clearly separating the 
five decisions that determine who is to be served, with what services, and 
for placement into which jobs, we can then examine the remainder of the 
program for influences on these decisions. 

Conversely, if we were to define planning as extending over all management 
decisions, we would lose focus on the essential questions unique and basic to 
employment and training matters, and cross into what would become a wider 
study of program management. The concrete implementation of decisions, for 
example, is not considered in this analysis as part of planning itself, although 
implementation is without doubt closely related to planning and in fact often 
performed by the same staff. 

Management decisions include all planning decisions, as well as other 
decisions necessary to organize, implement, and operate an employment and 
training program. Examples of management decisions include those categorized 
as planning decisions and other decisions such as arrangements to manage 
actual or potential conflict; choice of nature and use of monitoring and 
evaluation of programs; and the nature of staff relations with such key actors 
as political officials, advisory council members, and service deliverers. 
These management factors are important in considering planning because they 
form part of the context of planning decisions and thus may influence them. 

General context consists of conditions external to the CETA program and 
generally not open to short-run change through the actions of the local program 
staff, although some of them may be manipulable in the long run. Examples of 
such factors are economic conditions, the history of employment and training 
programs in the locality, and the structure of local government. The general 
context is important insofar as it influences the nature of planning. 

Our purpose, at the most general level, is to explain how and why prime 
sponsors plan. We began with the belief that the nature of prime sponsor planning 
is influenced by two factors: the general context and general management 
decisions. Testing this assumption led us to pose three central questions, 
which are discussed in sections that follow: 

1. What major variations exist in the nature of planning decisions? 

% 

2. Which factors in the general context and in the set of management 
decisions are important influences on the nature of planning and 
which are not? 

3. How do the important factors influence the nature of planning? 

10 



B. MODELS OF PLANNING 

To associate contextual factors and management decision factors with 
differing methods of planning, we first analyzed the nature of planning in 
the twelve sites we studied. 

Five dimensions of variation were utilized and applied to each of the 
five planning decision areas. Selection of each of these five variables 
was based on preliminary analysis that considered a wide array of planning 
.system characteristics and selected those that occurred most frequently and 
with greatest significance. 

Stability of relations among actors, the first dimension, was analyzed 
according to which actors participated in each decision; the extent and 
consistency of their influence; and the reasons for their influence* Overall 
stability across decision areas was also examined. 

Conflict over planning decisions was similarly analyzed, considering not 
only the level of conflict but also how well conflict was managed and channeled 
to productive ends. We observed that most sites had declined in overall level 
of conflict when reports for FY 1978 were compared to reports for 1974 and 1975, 
when local decision-making patterns were being established. But there was some 
variation in 1978 too. 

Degree of routine in planning is a summary measure that takes into account 
the presence of decision making routines and their success. Such routines 
included regular use of RFPs, substantial reliance on the advisory council, 
and the development and use of standard data analysis formats. 

Feedback mechanisms were assessed along two lines: whether they were 
present and whether they worked. They were interpreted to include both hard 
(for example, MIS, formal evaluation) and soft (for example, reliance on informal 
reports from job developers) variations. 

Deliberate attention to long-range decisions, the final dimension, was 
defined as decision-making that looked beyond the 1-year time span. To be 
assessed positively on this measure, a prime sponsor was required to display 
regular allocation of program resources to the function. 

On the basis of the major variations we observed, we constructed three 
general models of planning systems. We labeled these three models Crisis 
Management planning, Operations Management planning, and Future Oriented 
planning. The broad relationships among these three models are summarized 
in Figure 1. 

It should be noted that these models are derived from 12 prime sponsorships 
that were reputed to take planning seriously. In our other studies we have 
come across a number of sites in which planning is quite rudimentary and, in 
many important senses, nonexistent. Thus a fourth "model" is implicit: No 
Consistent Planning. However, since we are dealing with sites in which at least 
some consistent planning was observed to take place, we limit ourselves to 
discussing the contexts for and results of these three models. But we are 
not asserting that all prime sponsorships in the country necessarily fall into 
one of the three categories. At minimum, a number would fall into the No 
Consistent Planning category. Also, there may well be a higher proportion of 
Crisis Management prime sponsorships in the country than is indicated by our 
identification of only 1 of our 12 sites in that category. And there is 
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Figure 1: CHARACTERISTICS OF PLANNING MODELS 



probably a smaller proportion of Future Oriented sites than is indicated by 
our identification of 25% of our sites fitting that model. In short, we do 
not claim that the breakdown of our 12 sites reflects the breakdown among all 
460 prime sponsorships* In fact we would guess that the proportion of No 
Consistent Planning sites is substantial, the proportion of Crisis Management 
sites is larger, and the proportion of Future Oriented sites is smaller. 
This would also mean a smaller proportion of Operations Management sites. 

In t * ie Crisis Management model the influence structure is highly 
unstable. Actor influence changes from moment to moment * and from decision to 
decision. Little trust and stability in communication are present. In addition, 
when conflict arises, which is often, it is unmanaged (that is, there are no 
routines established and accepted for conflict resolution) . A critical 
characteristic of the Crisis Management system is the malfunctioning of the 
information feedback system. While good qualitative and quantitative information 
may in principle be available, without a feedback pipeline the information will 
not be utilized to make important decisions. In sum, the Crisis Management 
planning system is characterized by a reactive decision-making process that 
focuses, of necessity, on unpredictable problems, often organizational in 
character, that continually emerge from an unstable environment. No patterns 
of influence are established among actors, and a generally chaotic system seems 
to prevail. But, even in the midst of turmoil, planning is not abandoned. 
This fact creates a model of planning, instead of a model of non-planning. 

In contrast, the Operations Management system is characterized by a 
stable and well-oiled influence structure in which actors recognize their own 
and other actors 1 influence positions. In addition, conflict is well managed 
(that is, routines for resolving conflict are established, accepted, and 
practiced). Planning activities, whether qualitative or quantitative, 
generate important information, organize it, and communicate it to the important 
decision-making actors. A feedback operation is in place and is utilized. 
Finally, the decision-making process focuses mainly on short-term performance 
with a time frame usually a year or less in length. 

The Future Oriented planning model differs from the Operations Management 
model only in its deliberate attention to long-range decision areas. Future 
Oriented planning systems will not necessarily have any different or better 
monitoring systems. Both qualitative and quantitative information is gathered 
and fed back to important actors so as to improve ongoing programs in an 
incremental fashion. Yet, the Future Oriented system commits significant 
organizational time, energy, and capital, both human and financial, into the 
mapping of strategies for longrange goals covering a number of years in the 
future . 

Implicit in the nature of our models is the judgment about what characterizes 
the quality of planning systems. Clearly, the Crisis Management model charac- 
terizes a system that all prime sponsors wish they could avoid. Unstable 
influence patterns, unmanaged conflict, and inoperative feedback processes are 
all marks of a system in chaos that can have little hope of fulfilling the 
goals of a CETA program. However, while prime sponsors would certainly rather 
be in an Operations Management or a Future Oriented system, it is not clear 
whether the Future Oriented model is necessarily preferable to the Operations 
Management model in all cases, contexts, and times. It may be that at a certain 
point in the prime sponsor's development most long-range goals have been 
sufficiently accomplished and the continued extension of organizational 
resources into an ongoing Future Oriented part of the staff may be inefficient. 
The prime sponsor may make the best use of resources by just focusing on 



ongoing programs within annual plans. This does not mean that at some future 
point it would not be efficient and important to add the features of Future 
Orientation to the existing accomplishments of the Operations Management model, 
hold the Future Orientation in place long enough to inject additional planned 
change into the system and institutionalize it, and then revert to the 
Operations Management of the now institutionalized gains. The latter two 
models in this interpretation would vary in preferability according to the 
needs and situation of the prime sponsorship. 

There are not only one or two "good" sets of planned activities and 
ways of adapting to actor influence that should be applied across all prime 
sponsors, in all contexts. Some types of planning activities and some types 
of influence structures will be effective in some contexts while not in others. 
For example, a very sophisticated, highly quantitative monitoring system may 
be essential for good planning and decision-making in a large prime sponsorship 
with numerous service deliverers. Yet, in a much smaller prime sponsorship, 
one with only one or two service deliverers, a less rigorous monitoring 
system based largely on qualitative information may work quite well and 
resources may, in fact, be wasted on a more sophisticated system. However, 
if n monitoring system exists and there is no feedback to decision-makers 
on how well programs are operating, then a judgment can be made that the planning 
system is not functioning effectively. And to the extent that planning is 
important for good decision making, then the decision-making process could also be 
judged to be poor. 

C. EMPIRICAL BASIS OF THE MODELS 

The three models we have described have been characterized by variations 
along five dimensions. These five dimensions represent and summarize variation 
along two lines: differences by site and differences by nature of planning 
activities. The differences we observed will first be discussed by site, 
illustrating the characteristics of sites that are grouped according to 
their placement in the models. Differences will next be discussed by nature 
of planning activities. 

Differences Among Sites 

Crisis Management. Denver was the only site of our 12 that, in our 
judgment, clearly had a crisis orientation. At one time it had had a system 
more like the Operations Management model but it slipped into the Crisis 
Management model as the system underwent significant change. There has been 
high turnover in the professional staff concurrent with expansion of both program 
and staff size. This led to substantial demands on remaining staff, leaving 
little time for planning. Planning under these conditions was reactive and 
involved the resolution of the problems of the day. Subcontractors view the Denver 
Employment and Training Administration with substantial distrust, and there is 
little coordination of the subcontractors in what is intended to be a highly 
organized program. Thus, when conflicts do arise, they are often unmanaged, 
and the influence struggles that ensue indicate that influence patterns are highly 
unstable. There has been no regular monitoring and/or evaluation of performance. 
Finally, key staff are striving to build a stable system in which there is a 
noncrisis atmosphere; they are aware of their problems and are working hard to 
correct them. 

Operations Management* Eight of our sites fit the Operations Management 
planning model: San Francisco, Bergen County, Gulf Coast, Heartland, Atlanta, 
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Omaha 9 King-Snohomish, and Albuquerque. Some sites clearly fit this model, 
others less so, and still others seem to be in a state of transition to one 
of the other models. However, all eight sites had enough characteristics in 
common that we were convinced that they all approximated this model. 

San Francisco in some ways fits the Future Oriented model because it 
does acknowledge the importance of long-range planning. This concern, however, 
is more of an aspiration than reality. San Francisco, therefore, fits most 
closely into Operations Management. It could easily move into the Future 
Oriented model, given conditions that would facilitate such a shift (for 
example, more time for planning and a larger planning staff). 

King-Snohomish, on the other hand, is in a state of flux that suggests 
movement toward the Crisis Management orientation becasue of some upheaval 
at key staff levels. Thfe change in staff has resulted in the injection of 
some inexperienced actors into the planning process, which presented some 
problems but thus far has not threatened the viability of the system in place. 
Relations between key actors (service deliverers and staff) have historically 
been stable and remain reasonably so. Feedback has been provided, both for 
service deliverers and staff, by monitoring, and by continually developing the 
MIS. Conflict has been controlled by a strategy of incremental program changes 
and an elaborate decision-making process. In sum, although the consortium is 
experiencing some problems, the system has not become simply reactive to the 
problems of the day, but continues to perform in a manner best characterized 
by Operations Management. 

Other sites more clearly fall into the Operations Management model although 
the methods for achieving similar end results differ. For example, Omaha has a 
relatively sophisticated (in terms of use of quantitative data) monitoring 
system. In contrast, the Gulf Coast consortium does not have a rigorous 
monitoring system, nor does it appear to be necessary at that site; staff 
members are in frequent contact with service deliverers and handle most 
problems informally before formal monitoring visits are made. In Atlanta, 
monitoring is a routine, rigorously systematic operation. Bergen County, 
Heartland, and Albuquerque monitor in other similar ways. All sites in this cluste 
however, do provide feedback to service deliverers and staff (and other actors 
if relevant) so that adjustments can be made. In all cases it appears that, 
whatever the form of the monitoring procedure, good monitoring is essential 
to the Operations Management model. 

Conflict, when it occurs, is well managed in most Operations Management 
sites by careful planning by prime sponsor staff. The desire to manage conflict 
is frequently among the commitments of other key actors. In addition, all 
actors recognize their own influence position and accept some basic rules 
for conflict resolution. In San Francisco, for example, conflict emerges 
occasionally when service deliverer selection is being pursued; staff carefully 
builds its case based on quantitative data that give legitimacy to staff 
positions and can facilitate resolutions of conflicts. In Gulf Coast, the 
prime sponsor staff presents a "business like image" that seems to put the 
agency "above politics," and hence, probably minimizes conflictual situations. 
Conflict is managed in Heartland by anticipatory decision-making coupled with 
the administrative and political skills of the employment and training 
director. Other sites structure their planning systems to minimize the 
potential for conflict. 
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In all eight sites, critical actors, although they may vary in numbers 
and types, have established relatively stable techniques for interaction with 
one another, and various routines for decision-making have been worked out. 
Although there are some interruptions in these processes, there is relative 
stability in the decision-making system (including the planning process). 
The focus, in all eight sites, is on improvement of the system by making 
incremental adjustments. The primary emphasis is on short-term planning with, 
perhaps, occasional attention to a longer range future, although no regular 
and systematic attention is paid to it. 

Future Oriented. Baltimore, Penobscot, and Syracuse are most appropriately 
characterized by the Future Oriented model. They all manifest the characteristics 
of the Operations Management model, but they also are regularly concerned with 
issues that go beyond the 1-year time span. Baltimore expresses its concern 
with the future by maintaining a highly stable Operations Management system in a 
climate of awareness of the long-range effects of short-term decisions; an 
example of this concern is the establishment of the Labor Market Advisory 
Councils (which are representative of the private business community and could 
have long-term impact). The Penobscot Consortium consciously pursued long range 
goals when it made the decision to redirect the efforts of the employment and 
training system, and most actors continue to articulate long range goals as 
critical to the planning operation. In Syracuse, some staff members are assigned 
the responsibility of dealing with long-range concerns; the regular attention 
to economic issues is a key example. In all three cases, there are highly stable 
relations between critical actors who perform their responsibilities in a 
routinized, and, perhaps more importantly, accepted manner. Planning is closely 
linked with operations (although operationalized somewhat differently in 
Penobscot where the major portions of the program are run in-house). These 
characteristics have effectuated an environment characterized by low conflict 
between actors, and therefore, the freedom to engage in future oriented 
concerns. 

Generalizability to Other Sites. Given the purposive nature of our site 
selection (described in Section I) it was not unexpected that most of the sites 
cluster within one model. The quality, details, and functioning of the 
planning systems we observed varied a good deal, however. Thus, to reiterate 
a point made earlier, we do not claim that our three models are necessarily 
exhaustive. On the other hand, not only do they fit the observations at our 
12 sites but they are also intuitively pleasing when placed in an unsystematic 
way against our experience in the 32 sites visited previously, 

Differences in the Nature of Planning 

We examined multiple dimensions of the nature of planning at each site. 
These dimensions can be broadly separated into two classifications. First, 
we examined differences in actors participating in planning decisions. Second, 
we analyzed the group of factors describing the manner in which each of the 
five decisions is made. 

In assessing the influence of actors we relied on the judgments of 
two-person teams that spent 2 weeks in each of the 12 sites conducting a 
large number of interviews, observing meetings, and collecting documentary data. 
We attempted to identify actors that had either a high degree of influence or 
a medium degree of influence. The residual category was little or no influence. 
Reasons for influence included (1) control over funds for services, (2) access 
to generating political support from an identifiable constituency, (3) control 
over or special access to information, (4) access to special forums such as the 



advisory council where pressure could be brought to bear on other actors, 

(5) strong interest in and commitment to achieving a specific decision, 

(6) personal skill at bargaining, (7) default (no one else was much interested) f 
and (8) control over a prior decision that pre-dfetermined the outcome of a 
succeeding decision or decisions. When differences in findings from this 
analysis are summarized, the results lead to judgments on the first dimension 

of the planning models, stability of relations among actors. 

In the second section we look at five decision areas: local goals, 
target groups, target occupations, program mix, and service deliverers. In 
commenting on each area for decision we have considered a number of characteris- 
tics of activity in that area: (1) degree of openness in the process, (2) the 
use of socioeconomic data, (3) the use of monitoring and evaluation data, 
(4) the use of political considerations, (5) the formality of the processes 
used, (6) the degree of conflict present, and (7) the degree of change in programs. 
These factors, when collapsed across the five decision areas into summary 
measures, lead to ratings on the degree of routine, use of feedback, conflict 
management, and long-range planning dimensions of the planning models. 

Influence Structure Patterns. It is worth noting that, in general, the 
only actors actively engaged in the planning process at any site were prime 
sponsor staff, service deliverers, advisory councils, and political officials. 
These actors were present or absent in different configurations and with different 
strength in different locations and different decision areas. This generalization 
underscores the local nature of decision-making. In only one case was the 
employment service important in making any planning decisions and in no cases 
were representatives of the Employment and Training Administration important. 

We found, as we expected, that staff actors were generally most important, 
usually because of their control over information. Important variation existed. 
In the Crisis Management site, staff members were significantly less dominant 
than in other sites and service deliverers were very important. At the other 
extreme, staff members were active in all five decision areas in those sites 
we termed Future Oriented sites, and very seldom shared their influence with 
other actors. Service Deliverer influence, in addition to staff influence, 
was common in sites that we called Operations Management sites, where it 
clustered most significantly about the Service Deliverer Selection and Program 
Mix decision areas. Advisory council influence was found most frequently on 
the selection of target occupations and on selection of service deliverers, 
although the type and level of council influence did not strongly associate with 
other factors. Political official influence appeared in some areas but was not 
frequent. It should be noted that the infrequent participation of political 
officials in decision making may mask the greater indirect influence these 
officials actually may have because of anticipatory decision making by the 
staff. 

Planning Decision Areas. In the Local Goals area, we found a closed 
process in which staff based goals on value judgments with little use of data. 
Although there were variations on this pattern, they were not as numerous as 
variations in other decision areas. Future Oriented sites tended to have more 
specific long run goals, although they were not usually formally stated* 

The Target Group decision was most commonly the focus of attention by 
staff, service deliverers, and advisory council members, who made high use of 
socioeconomic and monitor ing /evaluation data. Seven of 12 sites shared 
a pattern of use of political considerations, service deliverer participation* 
use of formal process, and low use of monitoring/evaluation data (only one of 
these is a Future-Oriented site) . This pattern suggests that the Target Group 
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decision, because it directly affects allocation of resources, is next most 
controversial after the Service Deliverer Selection decision, and therefore 
receives serious consideration. 

The Target Occupation decision was treated at most sites as a technical 
one. Little non-staff participation was noted. Data were highly used. 
Substantial change in methods over time was noted, with change in three of 
the sites involving systematic introduction of business representatives into 
the decision. Two of these three were Future Oriented sites. 

The Program Mix decision split into two fairly clear patterns: one in 
which it was derived from other decisions and made with input from service 
delivery agencies, and another in which staff decided the issue and applied it 
to the Service Deliverer Selection decision. The first pattern was more 
frequent, with the second appearing in the three sites we labeled Future 
Oriented . 

The Service Deliverer Selection decision likewise displayed two patterns: 
three sites selected deliverers with low use of political data and a closed 
decision making process (the Future Oriented sites) while a majority of the 
others used political considerations in more open systems with high influence 
of service deliverers and advisory council members. 

D. THE EFFECTS OF CONTEXT ON THE NATURE OF PLANNING 

Why do prime sponsors plan as they do and not in some other way? Do prime 
sponsors have the flexibility to choose how they plan their programs, or is 
the nature of planning largely imposed by contextual conditions? Insight into 
these issues requires an examination of how context affects what gets planned 
and how it is planned. That is the purpose of this section. 

Complexity must characterize any investigation of the effects of context 
on the nature of planning because of two factors: the large number of possible 
contextual influences and the interrelationships among them. The first point 
is obvious. An illustration will clarify the second. The nature of the local 
economy and the quality of program staff are both contextual factors. Each may 
influence the nature of planning. Under certain conditions these two may cancel 
each other, leaving no net effect on planning. The net effect, however, is all 
that is apparent to the observer. Separating the interaction between the 
constraining effect of the economy and the facilitating effect of the good staff 
is not readily possible. Fortunately, the effect of any one factor is not of 
primary interest. We are, instead, concerned with identifying the pattern of 
contextual characteristics within Which "good" planning is found, and examining 
them in order to determine the feasibility of making interventions that extend 
"good" planning to greater numbers of sites. 

Investigation must therefore accomplish two tasks. It must select the 
most important from among many potential influences. Second, it must consider 
their net impact -on the nature of the planning system and explore how this 
impact is exerted. 

Selection of Contextual Variables 

We have defined the context for planning in air earlier section of this 
report. It will be remembered that the environment for five decisions that 



compose planning consists of two elements: general context and management 
decisions. The general context consists of conditions external to the program 
and not generally open to short-run change. Management decisions, because they 
determine the general climate within which the program is operated, are the 
second half of the environment for planning. This distinction is useful because 
factors in the general context are on the whole less subject to manipulation 
than management decisions. Selection of indicators to represent each of these 
elements was based on three principles. First, we examined factors that had 
been identified in previous research. Second, we analyzed factors that were 
mentioned during our site work with sufficient frequency to suggest their 
potential importance. Third, we have included some indicators to test the 
validity of certain operating assumptions offered to us by manpower practitioners. 
We are interested in each variable to the degree that it affected the nature 
of the planning system. (The effect of context on performance is examined 
separately in a later section.) 

General Context Elements 

Four groups of indicators (nature of local economy, nature of local population 
program size, and organizational nature) comprise those selected to indicate 
the general context. The nature of the local economy, a factor clearly beyond the 
immediate control of the program staff, is measured by three variables. The 
first of these is the local unemployment rate. To minimize the effects of 
variations over time, an average rate over five selected quarters (6-75 through 
9-77) has been used. Note has been made of which sites are Future Oriented (FO) 
and Crisis Management (CM) sites. Where no such note is attached, the site is 
an Operations Management site. As will be apparent, no strong pattern of 
association in this and other general context variables between the variable 
and the type of site emerges. In the present case, Future Oriented and Crisis 
Management sites are distributed over much of the range of unemployment rates. 



12 STUDY SITES ORDERED BY AVERAGE 
UNEMPLOYMENT RATE OVER 5 QUARTERS 
(6-75 through 9-77) 

Gulf Coast 4.3 

Omaha 6 . 4 

CM Denver 7.4 

U.S. AVERAGE 7.8 

FO Penobscot 7.8 

FO Baltimore 7 . 9 

Albuquerque 8 . 3 

Bergen County 8.6 

King-Snohomish 9 . 

Heartland 9.4 

FO Syracuse 9.4 

Atlanta 10.7 

San Francisco 10.7 

Growth in the local economy is the second indicator representing the nature 
of the local economy. Figures were derived from reports of the Bureau of Labor 
Statistics. They represent ratios of the number of jobs present in 1977 to the 
same number in 1970. Data for two sites (Penobscot and Heartland) were not 
available. 



10 STUDY SITES ORDERED BY GROWTH IN 
NUMBER OF JOBS IN THE LOCAL ECONOMY 
19701977 

Gulf Coast 45% 

Albuquerque 43% 

CM Denver 30% 

Atlanta 20% 

King-Snohomish 16% 

Omaha 16% 

San Francisco 10% 

FO Syracuse 8% 

FO Baltimore 7% 

Bergen 0% 

It is interesting to note that the two Future Oriented sites and San Francisco, 
the Operations Management site closest to Future Oriented status , had low growth 
rates; while Denver and King-Snohomish, the Crisis Management site and the 
Operations Management site most similar to it, enjoyed relatively high rates 
of growth, a counter-intuitive pattern* 

Simple aggregate measures of the number of jobs in an economy may mask 
influences that arise from the distribution of those jobs across various 
occupational sectors. To examine this possible contextual influence on the 
nature of planning, we obtained information from the BLS Employment and Earnings 
reports on the proportion of workers in local labor market areas who were 
employed in service occupations. (Service occupations were selected as an 
indicator for distribution of jobs because placement of CETA participants into 
these is typically less difficult than placement into other sectors.) Data 
were missing for Penobscot and Heartland. 



10 STUDY SITES ORDERED BY % WORKERS 

IN SERVICE OCCUPATIONS (1975) 

Albuquerque 24 

FO Baltimore 22 

San Francisco 22 

FO Syracuse 20 

CM Denver 19 

King-Snohomish 18 

Atlanta 18 

Omaha 17 

Bergen 17 

Gulf Coast 12 



It is clear that the range of variation is neither wide nor indicative 
of important relationships between distribution of jobs and nature of planning. 

The nature of the local population is clearly a potentially important 
contextual factor. Concentration of disadvantaged individuals in a prime sponsor- 
ship is, unfortunately, a factor without a thoroughly reliable indicator. 
Use of 1970 Census data must be rejected because of the remoteness of the 
observations and the known forces that have modified both size and relative 
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conditions of populations since then* The local unemployment rate offers 
some approximation of the nature of the local population that the program must 
serve o Although basically unsatisfying, this substitute does, when high, 
suggest that the sponsorship must serve both cyclically and structurally 
unemployed persons, and when low, that it must respond to the dominant presence 
of the structurally unemployed. We reference it to supplement our chief indicator 
of the nature of the population as a factor in the general context. This 
variable is the proportion of applicants at Employment Service offices who 
were economically disadvantaged for the period ending 9-77, as reported through 
the ESARS system. This indicator does have the shortcoming of not estimating 
the incidence of the characteristic for all persons in the jurisdiction, but 
does estimate it for a significant portion of the universe of need. Data 
were missing for four sites (Atlanta, Gulf Coast, Heartland, and Denver). 
Those for the other eight are reported on Table 3. 

The size of the program is clearly an element of the general context because 
it is determined by a formula that is beyond the influence of local staff. 
(We consider such optional programs as YIEPP and STIP separately in a later 
section.) Choosing FY 1978 Title I allocations as the indicator of size, 
and representing these in millions of dollars, we note that sites ranged in 
size from very large to very small, and that sites in any particular model of 
planning are not characterized by their size. 



12 STUDY SITES ORDERED BY SIZE OF 
FY 1978 Title I ALLOCATION (in millions of $) 

FO Baltimore 15.22 

King-Snohomish 12 . 10 

San Francisco 7.77 

Bergen County 5*61 

Atlanta 5.26 

CM Denver 3.68 

Omaha 3.26 

Heartland 3.19 

Albuquerque 3.06 

Gulf Coast 2.17 

FO Syracuse 1.66 

FO Penobscot 1.33 



Organizational characteristics may be important in both the general context 
and in management decisions as context. Two elements that, because they are 
beyond the power of staff to influence in the short run, are elements of the 
general context are 1) whether the sponsorship is a member of a consortium, 
and 2) its organizational location. Type of sponsorship has potential importance 
because of the complications introduced in a consortium by the presence of 
extra actors and priorities. Type of organizational location may become 
important to the extent that the parent body imposes constraints on the operation 
of the CETA program. Organizations outside city government, for example, 
frequently are free from constraints imposed on sponsors located within govern- 
mental structures. These latter often must abide by civil service requirements, 
purchasing procedures, and the involvement of city councils or other elected 
officials. Two tables are therefore presented to summarize the status of each 
sponsorship on these dimensions. 
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12 STUDY SITES BY TYPE OF 
SPONSORSHIP (FY 1978) 

CONSORTIA SINGLE UNIT 

Albuquerque Atlanta 

FO Baltimore Bergen County 

Gulf Coast* CM Denver*** 

Heartland San Francisco 

King-Snohomish FO Syracuse 
Omaha 
FO Penobscot* 



* Split from Balance of State Sponsorship after FY 1975. 
*** Consortium dissolved after FY 1975. 



ORGANIZATIONAL LOCATION OF ADMINISTRATIVE 
UNITS OF 12 STUDY SITES IN FY 1978 

City or County Office Consortium Community 

Executive Board Action Program 

Albuquerque Gulf Coast Bergen County 

Atlanta King-Snohomish 

FO Baltimore FO Penobscot 

CM Denver 

Heartland 

Omaha 

San Francisco 
FO Syracuse 



It is apparent that status as a Future Oriented or Operations Management 
site is not strongly associated with consortium membership. Likewise, the 
organizational location of a sponsorship has not displayed clear relationship to 
the type of planning conducted at the site. 

General Context Effects 

Table 3 arrays the factors of the general context by site, and clusters 
sites by models of planning. The most important conclusion that emerges from 
inspection of this table is that to the extent that the identified variables 
accurately summarize the important elements of the general context that might 
affect the nature of planning, these effects are minimal. An almost random 
pattern characterizes the distribution of these elements in the twelve sites, 
with one exception. All three Future Oriented sites received low ratings on 
the measure of economic growth. All three were lower than all other sites except 
for Bergen County. This puzzling pattern suggests that a relationship exists 
between low growth and the nature of planning. Such relationship might exist 
if local concern over limited growth stimulated active responses from CETA 
and other local programs, responses not called forth in other sites more comfor- 
table with their growth rates. Interviews at one site inconclusively supported 
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this possibility . However , in the absence of stronger evidence the pattern 
must be simply noted as an exception to the dominant finding that none of the 
models are strongly associated with a pattern of variables in the general 
context . We note that Operations Management sites cover the range of 
unemployment rates, include high and low growth sites, are not necessarily 
large or small, may be in a consortium but are not always, occur in 
populations that vary in degree of economic disadvantage, and have varying 
organizational locations* We are cautious of making inferences about the 
association of the Crisis Management model with factors in the general 
context because only one site provides observations and these do not 
conclusively establish a pattern. We do observe, however, that the characteristics 
of Denver are not distinguishable from those of the eight Operations Management 
sites. 

Management Decisions as Context Elements 

We next examine the importance of another aspect of context, that of the 
effect of general management decisions on the shape and nature of planning. 
Four important groups of indicators were selected. The first provides an 
estimate of the importance of staff in shaping the nature of planning by 
considering two variables, the quality of key staff and the presence of discon- 
tinuities in key staff tenure. The second takes into consideration the nature 
of relations between the staff and other actors. Important groups considered 
are political officials, service delivery contractors, advisory councils, and 
business representatives. The third considers some measures of the nature of 
data obtained for use in decisions, centering on the quality and extent of 
monitoring and evaluation information. Finally, we assess the extent to which 
optional program activity has influenced the nature of planning. 

The ability of staff may be among the most important of all contextual 
variables. However, it is a difficult matter to measure or even assign to 
nominal categories. Two approaches were used in recognition of this. First, 
field teams estimated average quality of key staff on the basis of observations 
and the judgments of interviewees. Second, turnover in key staff was tabulated. 
The results of the assessment in no case led to a site receiving lower than a 
medium-high judgment on quality of key staff. Those sites receiving high ratings 
were: 

Future Oriented 
Baltimore 
Penobscot 
Syracuse 

Operations Management 
Bergen County 
Omaha 
San Francisco 



This classification is applied with caution, not only because of the 
estimation problem, but also because of questions possible about causation: 
did a good site highlight the abilities of staff and did a difficult site 
tjend to obscure them? The second issue, it can be noted, is partially can- 
celled by utilization of the judgments of on-site observers. 

Discontinuities seemed highly important in the judgment of our site teams. 
The sole Crisis Management site (Denver) had experienced no less than four 
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different directors during its life, as well as much change at lower staff levels. 
King-Snohomish likewise had suffered major disruption from change in administratioi 
director, and upper level staff, and was the Operations Management site that 
displayed some characteristics resembling those of Crisis Management* 
Albuquerque , however , had changed directors with less disruption. 

The second major group of management variables covers the nature of the 
relations of staff members with other actors* Actor types most comnonly having 
influence on planning decisions in our twelve sites were staff members, 
political officials, service deliverers, and advisory council members. The 
patterns in which these actors participated in planning decisions were 
tabulated by assessing the influence of each actor type on each of four decision 
areas (target groups, target occupations, program mix, service deliverer 
selection) at each site. Separating actors with high and medium influence 
and summing the number of decision areas on which they had high or medium 
influence results in an approximation of the influence pattern* The findings 
are judged to represent adequately the approximate relative influence exerted 
by service delivery contractors and advisory council members. They do not, 



NUMBER OF DECISION AREAS IN WHICH ACTOR HAD 
HIGH OR MEDIUM PARTICIPATION I/ 



Political 
Officials 



Staff 
Members 



Service 
Deliverers 



Advisory 
Council 



Future Oriented 
Baltimore 
Penobscot 
Syracuse 

Operations Management 
Albuquerque 
Atlanta 
Bergen 
Gulf Coast 
Heartland 
King-Snohomish 
Omaha 
San Francisco 



1 
2 



4 
4 
4 



4 
4 
4 
4 
4 
3 
4 
4 



4 
4 
3 
3 
1 
4 



1 
3 



3 
3 
3 

3 
3 



Crisis Management 
Denver 



I/ Maximum score is 4 if actor had influence on target group, target 
occupation, program mix, and service deliverer decisions. Local 
goal decision was not included in the analysis because it typically 
is made by staff only. 

21 Labor Market Advisory Councils were, however, important participants 
in Baltimore decision making. 
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however, assess the participation of business representatives in decision 
making because these actors do not participate in the same fashion as the 
others do. The summary table is also judged to capture inadequately the 
importance of political officials because the influence of these actors may 
be more often exerted through anticipatory decision making by staff than by 
direct participation of the elected officials. Thus the table must be 
supplemented with further measures for these actors. 

Business involvement was the subject of specific inquiry by our field 
teams , who sought to determine which sites went beyond nominal appointment of 
business representatives to the advisory council and involved the private 
sector in either program operations or program planning. Those that were 
judged to have high business involvement were the three Future Oriented sites: 
Baltimore, Penobscot, and Syracuse. Omaha and Denver were judged to have medium 
involvement, chiefly because their programs included private-for profit 
service deliverers. The others had either minimal ties or no direct ties. 

As mentioned, the influence of political officials on Title I decisions 
is underestimated when only their direct participation in decision making is 
measured. In Atlanta, for example, the Mayor intervened in these decisions 
according to those we interviewed only three times since Title I began. 
To assume because his participation was in this sense limited that his 
influence was also limited would be to ignore that staff members make their 
decisions with awareness of the Mayor's needs and interests. They anticipate 
the Mayor's priorities and thus spare him the necessity of directly intervening. 
Partially as a consequence, they also enjoy his support for their decisions. 
Conversely, the lack of such political support may be expected to inhibit attempts 
by staff members to challenge the status quo. It is therefore appropriate to 
estimate which sites had direct access to their political superiors in ways 
that suggested the presence of higher than average political support for the 
Title I program. The following observations were reported by field teams: 



Albuquerque- 
Atlanta 



Current Mayor formerly directed Comprehensive 
Manpower Program. 

Mayor active as spokesperson for the disadvantaged, 
and in U.S. Conference of Mayors. CETA director 
active in Mayor's campaign while on leave of absence 
from program. 

Baltimore Mayor strongly supported positions taken by staff. 

Denver Former CETA director is now special assistant to 

Mayor for economic development programs. (He left 
to go into private law practice at the end of 1978.) 



Heartland 
Omaha 

Penobscot 
Syracuse 



County Commissioner very active in CETA decisions. 

CETA director was temporary head of Omaha Office of 
Management and Budget. 

County Commissioners strongly support CETA Executive 
Director. 

Mayor formerly president of U.S. Conference of Mayors 
and very supportive of local CETA efforts. 
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In our summary table (Table 4) we have indicated the presence of greater 
than average political support with a positive (+) sign and the presence of 
only average levels with a neutral (0) sign. We note that our present 
findings support our earlier findings (Ripley and Associates, 1977) that 
political officials tend to be much more involved in PSE than in Title I 
decisions. Although our present analysis does not assess political officials 1 
interest in PSE decisions, it does demonstrate their infrequent participation 
in Title I decisions, relative to other actors. We would, however, modify the 
finding of the earlier study that most officials believe the CETA programs 
cannot benefit them but may be potentially damaging and that they therefore 
encourage staff to function independently only so long as they do not foster 
any noticeable controversy by noting that in all of the Future Oriented sites 
and in some of the Operations Management sites, political officials have been willir 
to accept some controversy as the price of progress from the status quo to a 
more desired state. 

Many different kinds of data were used in planning by the prime sponsors 
we observed. To some extent, the use of data was associated with the ability 
of staff to dig it out, and is therefore represented by the variable estimating 
quality of staff. This tends to be especially true of "soft" data, those that 
are obtained by feedback through unofficial channels. There is in addition one 
kind of data that we feel to be essential to making informed planning decisions 
and that is available only if a management decision has devoted resources to 
gathering it: monitoring data. We therefore determined the extent to which 
sites were able to collect these data, and the quality of the information they 
gathered. The results are included in Table 4. We also assessed the extent 
and quality of evaluations in like manner, but do not include that analysis 
because it demonstrated that most sites attach far less importance to 
evaluation than to monitoring. 

Optional program activity, the final management decision we considered as 
a possible contextual variable, was defined as successful competition by a prime 
sponsorship for one of the grants awarded by DOL on other than formula allocation 
basis. We considered three possibilities because of the substantial work 
involved in preparing the applications and administering the grants, and 
because they were current at the time of our study: STIP, YIEPP Tier I, and 
YIEPP Tier II. Our interest was in both the possible direct effect of the grant 
in overloading staff with work and perhaps more importantly in the grants as 
representative of the general tendency of the sponsorship to undertake 
optional activity and devote sufficient resources to the effort to bring about 
success. Our appropriately coded findings anre reported in Table 4. It is 
noted that some YIEPP Tier grants are enormous over twice the size of the 
sponsorship f s Title I allocation in Denver and therefore likely to have 
important impact. 

Management Decisions as Context Effects 

Table 4 summarizes our findings for the variables we have discussed and 
organizes them by planning model and site. The general conclusion supported by 
this table is that strong associations do exist between the outcome of manage- 
ment decisions made by the prime sponsor and the nature of planning. More 
specifically, the Future Oriented sites are distinguished by a set of particular 
variations on the eight indicators considered. Future Oriented sites in this 
sample invariably have excellent key staff while only 3 of 9 Operations Manage- 
ment were judged to have excellent staff. No discontinuities in key staff 
or administration existed among FO sites. Influence structures are uniformly 
staff dominated with only minor service deliverer participation in determination 
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of planning decisions . (Advisory Councils may or may not be important*) The 
three FO sites have achieved business involvement to greater degrees than the 
QM sites . Above average political support is probably present. Monitoring 
is uniformly high in both extent and quality Finally , the FO sites display 
a common tendency to seek and win optional grants that complement the activities 
they support under their formula allocations* 

Operations and Crisis Management sites, in contrast, shared significant 
influence with service deliverers in 7 of 9 cases . Three were handicapped by 
discontinuities in key staff , in two cases severely. Few sought the 
involvement of business . Political support above average levels was probably 
present in only 4 of 9 cases, A majority did, however, win optional grant 
competitions 9 for the large Tier I YIEPP program in two cases. Only 3 of the 
9 possessed monitoring systems that we judged high in both extent and quality 
of the monitoring. 

Thus 5 although no contextual factor emanating from management decisions 
was unique to the Future Oriented sites , the pattern of staff -dominated 
influence structure, high quality staff, stronger than average political support, 
thorough monitoring, and active involvement of business was characteristic. 

We are hesitant in offering observations on the differences that 
characterize the Crisis Management model because only Denver fell into it, 
thus precluding observation of patterns. We do note that Denver had an 
unsettled influence structure in which service deliverers are very important 
and staff is much less important than average. Denver has also experienced 
major discontinuity in key staff positions and is attempting to cope with high 
levels of optional program activity. Political support, although present, has 
tended to constrain rather than facilitate. Staff quality is not uniformly 
high. Monitoring is very weak. On the basis of these observations, we hypothesize 
that the Crisis Management model may be associated with contextual factors 
emanating from management decisions as strongly as is the Future Oriented 
model. Naturally, the content of the relationship is reversed. However, this 
point should not be regarded as established given our limited data base* 

An interesting anomaly is offered by the presence of Omaha in the Operations 
Management model although its ratings on the individual variables are with 
the exception of the optional program activity category in the pattern of the 
Future Oriented sites. However, the departure of large meatpacking firms from 
the Omaha area may be a unique local problem that because it has reordered 
the local economy from an industrial to a service-based one has absorbed the 
attention of program staff. 

The Effects of Context Summary 

We initially posed the question of how context affects the nature of 
planning. We next separately assessed two basic types of context: the general 
context and the context formed by management decisions. We found that the 
general context is not strongly associated with the type of planning model, 
but that the context created by management decisions is so associated. This 
is particularly significant because the majority of the management decisions 
we have examined are manipulable. We hasten to add the recognition that such 
manipulation can be extremely difficult and require major efforts. But it 
may be possible if those wfto desire it are willing to pay the price that may be 
required. We further recognize that a site attempting to convert constraining 
patterns among the management decisions to favorable ones may be handicapped 
to the extent that it must devise a number of responses at once. The more 
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constraining factors the staff must respond to, the less likely it is that 
each response will be successful because of limits on staff accomplishments 
simply imposed by available time. Likewise, the greater the number of 
constraining contextual factors present at a site, the greater is the 
likelihood that a response designed to counter one constraining feature will be 
crippled by another contextual factor and fail to have the desired result. 
For example, an BFP system designed in one of our sites to respond to a 
stressful service deliverer environment failed in part because staff 
discontinuities, another contextual feature, led to a non-functional MIS 
system, a third contextual feature, the lack of which doomed the KFP effort 
and left planning decisions essentially beyond control. (Other factors were 
also important.) Finding a starting point to escape a counter-productive 
pattern may be indeed difficult. 

On the positive side, we observe that although difficulties exist, they 
are not insurmountable. The key elements we have identified that characterize 
the Future Oriented sites are straightforward: 

Key staff of the highest quality, who are encouraged to remain 
with the program and offer it consistent leadership. 

Monitoring systems that yield accurate and complete information 
about the program. 

Control of service deliverer participation in key decisions that 
affect who the program serves and how it serves them. 

Deliberate steps to involve the business community in more than 
token activities. 

A willingness to look beyond the narrow boundaries of operating 
programs allocated by formula (indicated by the presence of optional 
program activity). 

Political officials who support decisions made by staff and are 
willing to accept limited controversy when it is the price of progress. 

With the exception of the last two points, procedures to develop strengths 
in these areas either exist or are being developed. Management literature offers 
methods to recruit, train, and motivate staff. It also provides procedures 
for developing and using monitoring systems. The experience of many prime 
sponsors is available as a guide in controlling service deliverer participation 
without loss of the employment and training knowledge these deliverers have 
accumulated. Use of non-voting service deliverer groups auxiliary to the 
advisory council, of RFPs and EFQs, of fair and open decision-making rules 
centered on objective data, and infrequently of direct prime sponsor operation 
of programs are all examples. Business involvement once rare is the current 
focus of experimentation under Title VII of the 1978 CETA Amendments. Although 
this experiment may yield limited success if economic conditions deteriorate, even 
these limited successes will represent considerable improvement over present 
low activity levels. Even a willingness to innovate and the support of 
political officials, although more subtle in origin than the other elements, are, 
by the evidence of our sites, within the abilities of a reasonably advantaged 
prime sponsorship. The combination of all these factors appears associated with 
a distinctive type of planning. Whether these factors are also associated with 
program performance is the question we will examine next. 
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III. EXPLAINING PROGRAM PERFORMANCE 



A. OVERVIEW 

Many factors in addition to the nature of planning will affect prime 
sponsors 9 program performance. While that is a simple statement, even an 
obvious one, it quickly becomes mired in complexity when the range of factors 
that may impinge on program performance is enumerated. Prime sponsor staff 
administer local manpower programs in complex and diverse local environments 
or contexts that include economic conditions, demographic characteristics, 
political conditions, previous program history, Department of Labor guidelines, 
and attitudes and preferences of relevant actors. In addition to contextual 
features such as these, characteristics of the staff's own organization and 
its management decisions often have an impact on performance (manpower planning 
activities and decisions are considered as a subset of staff management 
characteristics or decisions). Characteristics of the manpower delivery 
system in operation, and the ability to implement planning decisions can 
obviously also have an effect on program performance. Adding to the overall 
difficulty in understanding what affects program performance is the ambiguity 
of the nature of program performance itself. It may be thought of in terms 
of short-term outcomes like placement rates, wage gain after CETA participation, 
and cost efficiency measures. It may be more appropriately measured in terms 
of long range economic impacts on former CETA participants. And performance 
can also be measured in terms of administrative and process oriented features 
such as degree of participation in decision-making by non-staff actors. 

Figure 2 presents a diagram to help visualize the various relationships 
that are possible between performance factors and antecedent conditions. This 
diagram will be used to organize the present investigation into explaining 
program performance. The shading of sections for context, management, and 
planning indicates that all of these areas affect performance. 

In Section II, we examined associations between selected features 
of local context and planning models, as well as linkages between management 
characteristics and planning, to help explain under what conditions different 
types of planning would emerge. 

In the present Section, we want to examine the linkage between planning 
and performance. But because performance can be affected by many factors other 
than planning, we want to look at the linkages between features of context and 
performance, as well as associations between management characteristics and 
performance. We will examine again some findings about the impact of program 
mix and participant characteristics on performance reported in earlier 
research (see, for example, Ripley and Associates, 1978: chapters 4 & 5; 
Mirengoff and Rindler, 1978: chapter 9). We are also interested in testing 
the validity of certain operating assumptions (conventional wisdom) of 
manpower practitioners. The analysis in this section will focus on describing 
and explaining the linkages found for the 12 prime sponsorships included in the 
areawide planning study. Where appropriate, we will also compare our findings 
about the 12 planning sites with the situation in the 30 other prime sponsorships 
in which we conducted field work for two previous studies. 
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Discussion of Performance Measures 

Unfortunately 9 there is no single summary measure of prime sponsor 
performance that would allow a researcher or evaluator to classify prime sponsors 
into groups of good or bad, worst or best performers. Such a summary classi- 
fication would be too simplistic anyway, because "performance" is a concept 
with many dimensions* Performance measures can include placement rates , quality 
of placements ? participants 1 wage gain in the short run, wage gain in the long 
run, economic impact of CETA on the local community, management efficiency 
of the prime sponsors 1 administration, cost efficiency of manpower programs, 
local prime sponsor perceptions about success in achieving local goals, and 
many more. All of these are legitimate aspects of performance, but there are 
not equally important, and furthermore, data are not equally available to 
measure each aspect of performance. 

The impact of CETA participation on the future employment, earnings and 
career advances of participants impact that is categorized as long range 
program outcomes in Figure 2 is, or at least should be, the ultimate measure 
of prime sponsor performance* Improvement in the economic situation of CETA 
participants increased earned income is a major purpose of the CETA program, 
as Section 2 of the 1978 amendments to the CETA legislation clearly indicate. 
Ideally, if success is to be judged in terms of economic impact on the 
participant, data should be available to measure not just whether a person 
was placed after CETA, but also to measure long-run economic consequences such 
as retention in employment, wage gain over time, and client satisfaction with 
CETA services. The only source of such data, however, is the Continuous 
Longitudinal Manpower Study (CLMS) funded by the Employment and Training 
Administration, but its findings based on followups of a nationwide sample of 
participants are just beginning to emerge (see Westat, 1978). 

Prime sponsors have not been required to collect systematic data on the 
post-CETA experiences of participants. Followup on participants at the prime 
sponsor level is almost totally absent. This results in a lack of knowledge 
about quality of placements, type of occupations, long range wage change of 
participants, and moves up (or down) career ladders. A few prime sponsors do 
conduct this type of impact evaluation on their own, but most do not. The 
federal quarterly report forms do not contain space from prime sponsors to 
report on participants 1 post-CETA experiences even as an optional item. 

Thus even though long term economic outcomes on participants may be 
recognized abstractly as the most desirable way to measure prime sponsor 
performance, data are not available to allow comparisons and judgments about 
prime sponsors to be made. 

The existing data collection system established by the Department of 
Labor stresses only selected short run quantitative measures of performance 
such as number of participants enrolled, placements obtained, and nonpositive 
terminations for all program activities within a single title. Information 
on participant termination experiences by individual program activity (classroom 
training, OJT, work experience) is not included, however, and comparisons of the 
effectiveness of different types of program strategies therefore cannot be made. 
A few prime sponsors do produce comparative information at their own initiative. 
Most do not because it is not required by the Department of Labor. And even 
those that do collect such data cannot report it to DOL because the quarterly 
report forms do not contain space for such optional information. 



In short, the existing data on prime sponsor performance capture only a 
limited dimension of performance, and permit only rudimentary judgments to be 
made. As Mirengoff and Rlndler note (1978:222-223) : ff o.. the present data 
system. ..has gaps in essential data, a lack of flexibility for making 
cross tabulations, and poor quality control." 

In this report, we can note the shortcomings of the data available from 
quarterly reports, but we must use it anyway because they are available, and 
because they reflect the main goal of the Department of Labor for CETA Title I 
during its first four years, namely placements. Discussion of the short term 
program performance measures used in this analysis follows. 

How effective is participation in CETA in terms of getting people into an 
unsubsidized job? The overall placement rate (abbreviated PLRATE) indicates 
what proportion of the total participants enrolled succeeded in obtaining 
unsubsidized employment. It is computed by dividing the number entering 
employment by the number enrolled for each quarter. 

Because the number of people who enter employment is comprised of self- 
placements and direct placements (persons who received just outreach, intake, 
manpower services and perhaps supportive services) as well as indirect placements 
(people who receive training and employment experience as well as outreach, 
intake, manpower services and perhaps supportive services), it is useful to 
include an additional measure of placement that taps the effectiveness of the 
real essence of CETA services (the training and employment experience) in 
getting participants a job. The number of indirect placements divided by the 
total number of participants enrolled provides one plausible measure of CETA's 
placement efficiency (abbreviated PLEFFIC) , and indicates what proportion of all 
the people enrolled got unsubsidized employment after receiving CETA training or 
employment experience (that is, CETA services other than just outreach, intake, 
or manpower services). 

The rate of nonpositive terminations (abbreviated OTT) from Title I 
indicates the proportion of participants who leave the CETA program for other 
than positive reasons (such as obtaining employment or returning to school or 
entering the military.) A high nonpositive termination rate can indicate 
problems in management of recruitment, assessment, and referral of enrollees, 
as well as problems in participant flow and placement activities. NPT is 
computed by dividing the number of nonpositive terminations by the number of all 
terminations for the quarter. 

The cost per enrollee ($ENROLL) and cost per placement ($PLCMT) provide 
two simple measures of the economic efficiency of the operation of local CETA 
programs. The measures are computed by dividing the total accrued expenditures 
by the number of enrollees and the number of persons placed, respectively. 

We should explain the decision not to use one category of data that is 
included in the quarterly reports submitted by prime sponsors. The Quarterly 
Summary of Participant Characteristics (QSPC) includes reporting sections that 
compare the wages of participants placed in unsubsidized employment after CETA 
with the wages they earned before CETA. We chose not to use this information 
to construct a wage gain measure for several reasons, including serious doubts 
about the accuracy of the data reported, and lack of controls for the effect 
of inflation over time, and the lack of a control for different prevailing 
wage rates among prime sponsors. The most troublesome of these drawbacks was 
the concern over the quality of the data, however. Interviewees indicated that 
the figures reported tended to be estimates at best, and that they also tended 
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to overstate the actual wages earned after placement* Verification of the 
reported figures is not possible 9 either for prime sponsors or for researchers 
such as ourselves* Examination of the completed QSPC forms makes it clear that 
prime sponsors 1 MIS staff do not understand how to complete the pre- and post- 
CETA wages section of the forms numbers that are supposed to add up don't, 
and large numbers of placements are unaccounted for. Given these problems, we 
reluctantly excluded wage change from our performance measures. 

The concept of standardized performance measures for CETA was developed 
and articulated by the Department of Labor in FY 76. Many prime sponsors objected 
to the manner in which the performance measures were applied 9 arguing that local 
diversity of clients, programs, and economic conditions rendered uniform 
national standards arbitrary and inapplicable. e have not attempted to set 
arbitrary ranges of acceptable or unacceptable performance for the measures 
discussed above. We have some doubts about the validity of some of the 
performance measures and about the quality of data reported in some cases. 
However, we used these measures in a previous study with results that did no 
violence to more qualitative judgments (Ripley and Associates, 1978). We have 
also avoided overreliance on any single measure as a safeguard against making 
invalid inferences. 

Methodology 

We relied on a variety of techniques to describe and explain relationships 
among the explanatory and performance measures. We used simple descriptive 
statistics and contingency tables when the data categories were only nominal 
or when the number of data points (observations) was too small to permit any 
other kind of technique to be used. We used bivariate correlational analysis 
to test for the strength of associations between pairs of variables, and we 
used step-wise multiple regressions to test for the effects of numerous 
explanatory variables on performance measures.* 

Many of the quantitative data used to measure explanatory factors (for 
example, unemployment rate, program expenditures, participant characteristics) 
were available from prime sponsors' quarterly reports to DOL (Quarterly Summary 
of Participant Characteristics, Program Status Summary, and Financial Status 
Report). We used data from these reports for the quarters from December 1974 
through June 1978 (the most recent quarter for which data was available). Many 
other explanatory factors that we investigated were more qualitative in nature 
(for example, attitudes, preferences, and management characteristics of the 
prime sponsorship). For these more qualitative measures, the time points used 
in analysis were limited to those quarters during which field work was done, 
plus the quarters immediately preceding and immediately following the on-site 
quarter. The reason for this restriction is simply that we could not be 
confident that the aspects of prime sponsor characteristics captured in the 
qualitative measures would remain constant over time. (In fact, it is the 
nature of these variables to change; we could judge them for the period of 
time of on-site field research, but we did not feel comfortable extrapolating 
forward into the future or backward into the past for more than one quarter.) 



* Correlation and multivariate analyses used the SPSS (Statistical Package for 
Social Scientists) programs, which are supported at the Instruction and 
Research Computer Center at Ohio State University. 



B. RELATIONSHIPS BETWEEN CONTEXT AND PERFORMANCE 

In this section we examine the associations between selected parts of 
the prime sponsors 1 local context and the standard indicators of performance* 
The strategy guiding the choice of context elements to be examined was based 
on a desire to re-examine previous research findings (Ripley and Associates, 
1978) using a different sample of prime sponsors as well as a desire to 
examine some of the common assumptions ("conventional wisdom") underlying 
manpower program operations- The availability of data of reasonable quality , 
accuracy, and comparability among prime sponsors also contributed to some of 
the decisions. In the following pages, the impact of economic conditions, 
demographic conditions, attitudes, and previous program decisions are 
examined, using both bivariate and multivariate analyses* 

Bivariate Relationships Between Economic Conditions and Performance 

Unemployment Rate. e used the local unemployment rate to measure economic 
conditions not only because it is a readily accessible measure, but also because 
it is widely perceived to be the most important feature of local economies that 
constrains CETA performance. Monthly unemployment rate data from the 
Bureau of Labor Statistics were averaged to obtain quarterly and annual averages 
of unemployment rate in the prime sponsorships. 

We wanted to examine the statistical association between unemployment 
rate and performance measures to determine whether and how much unemployment 
constrains good performance. If unemployment rate correlates strongly with 
performance measures , then manpower practitioners are correct in believing that 
unemployment rate will limit program choices and performance. If, however, 
there is no relationship or only a weak one, then local manpower practitioners 
would have greater latitude in making program choices and in managing program 
performance than they may realize. Specifically, the conventional wisdom that 
high unemployment is associated with low placement rates (and the converse, that 
low unemployment is associated with high placement rates) would be seriously 
challenged if no relationship or only a weak relationship were found. The 
results of an earlier CETA study did present such a challenge to conventional 
wisdom (see Ripley and associates, 1978:82-85). 

Using bivariate correlational analysis we tested the association between 
unemployment rate and performance in a number of ways. We correlated the 
quarterly unemployment rate with the cumulative performance measures for the 
same quarters. We also correlated quarterly unemployment with performance 
data that were lagged one and two quarters later, reasoning that the effects of 
unemployment may take some time to be observed in performance. We also correlated 
the annual unemployment rate (computed on a fiscal year basis, to be 
comparable with the performance data) with just the cumulative end of fiscal 
year performance figures, reasoning that relationship between non-cumulative 
unemployment rates and cumulative performance data might not be apparent unless 
the annual average were used. For each test using different operationalizations 
of unemployment rate, we examined all quarters since December, 1974, together 
for the 12 planning sites, as well as separating out the end of fiscal year 
quarters and analyzing them separately. 

The general conclusion emerging from all of the tests of unemployment 
rate and performance measures is that there were no strong relationships. 
Table 5 presents the strongest correlations found those between quarterly 
unemployment rate and program performance measures. The correlations between 
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annual and lagged unemployment rate and performance measures were all weaker 
than those presented in Table 5, and for space reasons they are not presented in 

tabular form** 

As Table 5 shows , the highest single correlation coefficient obtained 
was only .31 (between unemployment rate and nonpositive termination rate) for 
the end of fiscal year quarters, "When all quarters were analyzed together,, 
the relationship was weaker (.25). This suggests there was some tendency, 
but not a strong one, in the 12 planning sites for higher unemployment to be 
associated with higher nonpositive termination rates . 

There was also a weak relationship present between quarterly unemployment 
rate and both measures of placement for the end of fiscal year quarters in the 
planning sites . The weak inverse relationship (r f s = -.26 and -.27) suggests 
that there was a slight tendency for higher unemployment rates to be associated 
with lower placement rates. This supports the generally accepted belief about 
the effects of unemployment , but it should be noted that the relationships 
are very weak and ought not to be considered determinative. Certainly they do 
not justify an attitude that the prime sponsor staff are helpless to improve 
placement rates because of the level of unemployment rate. 

There were no correlations greater than the cutoff of .20 for either 
measure of cost efficiency, except for a weak inverse relations between 
quarterly unemployment rate and cost per enrollee for the end of fiscal year 
quarters. This relationship means that there was a weak tendency for prime 
sponsorships with higher unemployment rates to have lower costs per enrollee. 

A possible exception to this generalization can occur in a prime sponsor 
in which unemployment is rising rapidly, specifically in industries in which 
the prime sponsor has concentrated training and placements, in such a case, 
the impact of unemployment on performance would be strong and direct. But 
that possible exception does not weaken the general finding that unemployment 
is only weakly associated with performance. 

The weak to non-existent relationships found in this part of the analysis 
support the general assertion that program performance is not controlled by 
unemployment rates. The most that these results show is a weak inverse 
relationship between unemployment and placement and a modest positive 
relationship between unemployment and nonpositive terminations for the 12 
prime sponsorships in the planning study. 

We extended the bivariate analysis of unemployment rate to see whether 
changes in unemployment were related to changes in program mix measures in 
the 12 planning sites. We again found that unemployment rate was not strongly 
related to program mix either in terms of enrollments or expenditures. There 
was a weak and unexpected relationship between higher proportions of OJT 
expenditures and higher unemployment rates (r=.24), which is not an association 
that conventional wisdom would predict. Practitioner belief is that OJT 
expenditures would decrease in times of higher unemployment. 



* In discussing correlation coefficients, we used the following guidelines 
throughout Section III: coefficients of less than .20 were not considered 
to be strong enough to represent a relationship; those between .20 and .30 
were described as representing a weak relationship; coefficients between .30 
and .50 describe a moderately strong relationship, and those greater than .50 
represent a very strong relationship. 
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There was also a weak tendency for the proportion of work experience 
enrollments to be higher in prime sponsors with higher unemployment rates 
(none of the r ? s was greater than .25) 9 which is more in keeping with expec- 
tations. The proportion of "other 11 enrollments showed a moderately strong 
tendency to be higher in prime sponsors with higher unemployment (r=.40) 

More significant than the relationships found between unemployment rate 
and program mix measures, most of which were only weak, was the absence of 
strong relationships between unemployment rate and the proportion of either 
expenditures or enrollments for training activities. The absence of strong 
correlations means that program commitments to classroom training and OJT 
were not dictated by unemployment rates in the 12 prime sponsors in the 
planning study. Changes in expenditures and enrollments were occurring 
independently of changes in the unemployment rate. Unemployment rate was 
not a controlling factor determining the actual mix of program expenditures 
and enrollments. 

Other Indicators of Economic Conditions. Although unemployment rate is 
generally accepted as a good measure of local economic conditions, we wanted 
to use some additional measures of local economies to enlarge our investigation 
into the relationship between this context feature and program performance. 
Comparable labor market data were not available for two of the planning sites 
(Penobscot and Heartland) so they had to be excluded from this portion of 
the analysis. For the other ten sites, we obtained information from the BLS 
Employment and Earnings reports on the proportion of workers in local labor 
market areas who were employed in service occupations. We felt there might 
be a relationship between the proportion of services in an area and the prime 
sponsor f s placement rates. Specifically we hypothesized that placements would 
be easier and therefore placement rates would be higher in economies with a large 
proportion of service occupations because jobs in the services classification 
are generally more accessible to CETA participants than are jobs in 
manufacturing in which union memberships and skills requirements often pose 
significant barriers to employment of CETA participants. 

The results of correlations between the proportion of service workers 
and the placement rate measures showed that there was no relationship, 
however. The absence of a relationship is encouraging because it indicates 
that prime sponsors' placement rates were not tied to the prevalence of 
service occupations in their area, but rather were diversified among other segments 
of the economy. 

We also used BLS reports to derive a simple measure of economic growth in the 
planning sites, computing a ratio of the total number of workers in the labor 
market area in 1977 compared to the number of workers in 1970. The ratio ranged 
in value from 1.00 (indicating 0% growth in the number of jobs) to 1.45 (indica- 
ting a very large 45% increase in the number of jobs). (Penobscot and Heartland 
data were again unavailable.) 

Practitioner wisdom suggests that placement rates would be higher in 
prime sponsors with high growth rates. If the number of jobs is increasing 
it is presumably easier to place people. In areas with low or no growth, 
placements would presumably be more difficult. Our analysis, however, showed 
that growth in the economy is not a guarantee of good placement rates, nor is 
lack of growth a serious constraint to good placement performance. 



Three of the prime sponsorships with the lowest growth (Bergen had 0%, 
Baltimore had 7%, and San Francisco had 10% growth) in the number of jobs 
between 1970 and 1977 had very high placement rates (the overall placement 
rate for fiscal 1978 and fiscal 1977 was averaged) . Nearly all of the prime 
sponsors with higher growth in their economies had placement rate figures 
that were lower than these three prime sponsors with low growth . 

Of the two prime sponsorships with the highest growth rates (Albuquerque 
had a 43% increase and Gulf Coast had a 45% increase) , Albuquerque did show 
the highest overall placement rate of any of the ten prime sponsors analyzed. 
Gulf Coast, despite an even higher growth rate* had a placement rate that 
was good but not outstanding; it was in fact lower than the overall placement 
rates achieved by the three prime sponsors with the least growth , and the same 
as the placement rate of the fourth prime sponsor with low growth (Syracuse) . 

Other prime sponsors with high growth rates showed good placement rates , 
but not as sensational as one would expect if the effect of growth on CETA 
placements were really the key to success. None of the other prime sponsors 
with higher growth achieved placement rates as good as the placement rates of 
the prime sponsors with the least growth. 

Table 6 summarizes the data on growth in a number of jobs and prime sponsor 
placement rates. This table supports the general conclusion that while growth 
in the economy doesn't hurt placement rates, it is not an essential condition. 
Prime sponsors with low growth can still achieve very good placement rates 
although they may have to work harder to do it with measures such as more 
conscientious planning, supervision of job developers, and attention to 
occupational trends. 

Bivariate Relationships Between Demographic Characteristics and Performance 

We used characteristics of CETA enrollees as a surrogate measure of 
demographic features in the prime sponsorship. Earlier research has shown 
that the characteristics of CETA enrollees are reasonably representative 
of the universe of need in the prime sponsorship (Ripley and Associates, 
1978:54-55). 

An additional reason for investigating demographic characteristics of 
participants is that conventional wisdom among manpower practitioners suggests 
that the type of participant served determines the level of program success. 
e wanted to test the presumed linkage between this element of context which 
is manipulable by manpower staff through the decisions and policies of intake 
and recruitment agencies and program performance. We also wanted to examine 
an earlier finding of almost no relationship between participant characteristics 
and performance (Ripley and Associates, 1978:91-93). 

We examined both all enrollees and all participants placed, using a 
/ariety of characteristics typifying persons more "difficult" to serve and 
Dlace. These included: percent economically disadvantaged, percent welfare 
recipients (AFDC and public assistance), percent female, percent with less 
than a high school education or equivalent, percent who were unemployed at 
the time they entered CETA, percent nonwhite, and percent youth (less than 
22 years old) . The data came from the QSPCs . We examined the 12 sites in the 
planning study for all quarters except September 1974 and also for just the 
end of fiscal year quarters. 
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Table 61 Growth In Number of Jobs in the Local Economy f 1970-1977 , 

and CETA Placement Rates 



Prime Sponsor I/ 


Economic 
Growth 
1970-77 


Overall 
Placement Rate 
(PLRATE) 2J 


Bergen County 


0% 


40% 


Baltimore Csrt. 


7% 


40% 


Syracuse 


8% 


38% 


San Francisco 


10% 


40% 


Omaha 


16% 


36% 


KSMC 


16% 


30% 


Atlanta 


20% 


32% 


Denver 


30% 


35% 


Albuquerque 


43% 


48% 


Gulf Coast 


45% 


38% 



I/ Data not available for Penobscot and Heartland Consortia. 

2/ Cumulative overall placement rates for fiscal 1977 and 1978 averaged, 



Table 7 presents the results of correlations between participant 
characteristics and performance measures * (Data are reported only for the 
characteristics of all enrollees for end of fiscal year quarters.) The 
relationships between characteristics of participants placed and performance 
measures were much weaker (although none were contrary to those in Table 7), 
and for space reasons they are not included in the table. 

The results provide only limited support for the general notion that 
participant characteristics per se determine program performance levels, 
or that certain types of people cause poorer performance. Many of the 
associations between client characteristics and performance measures were absent 
or were only weak. For example, the percent of economically disadvantaged was 
not associated with either placement or nonpositive termination, nor was the 
percent female. The percent of welfare recipients showed only a weak depressing 
effect on only one measure of placement. 

None of the client characteristics were related to nonpositive terminations. 
Only the percent economically disadvantaged and the percent of welfare 
recipients showed a relationship with cost per placement, and that was extremely 
weak. 

Some of the results of the bivariate analysis between client characteristics 
and performance measures do support the conventional expectations, however. 
For example, there was some association between the percent of nonwhite 
enrollees and poorer placement rates. The most notable relationships were between 
the proportion of young enrollees and placement and costs per enrollees. 
For example, both the percent of enrollees who were less than 22 and the percent 
of enrollees who did not have a high school education showed a distinct association 
with lower rates of placement, for both measures of placement (r f s were in the 
-.40 to -.50 range). These characteristics were also related to cost per 
enrollee, though less strongly. There was no association between the youth 
measures and the cost per placement, however. (There was a high correlation 
between the percent less than 22 and the percent without a high school education, 
in the .9 range, suggesting strongly that both variables are measuring the 
same group of people in the 12 planning sites. This is logical, since many of 
the young enrollees in CETA programs are high school students who have not yet 
graduated.) 

Based on the bivariate analysis of client characteristics and performance 
measures, we can conclude that except for the younger enrollee group, the 
demographic characteristics of participants have an unimportant effect on prime 
sponsor program performance. This suggests that prime sponsor staff need not 
feel that the demographic composition of their participant group will predetermine 
the level of performance on placement, nonpositive termination, and cost effi- 
ciency measures. This also suggests that screening clients at intake and 
referral (skimming or creaming) is not necessarily going to result in better 
performance. 

A caution in interpreting these results of bivariate relationships must 
?e offered. We are not asserting on the basis of these statistics that there 
is no difference among clients, and that the nature of participants has no 
bearing on program performance results. Indeed, we believe that CETA 
participants, like most people, vary in terms of the personal motivation, 
incentive, and the barriers to employment .that they face. These types of 
qualitative characteristics can affect program performance, but they are 
not accurately measured by the demographic data reported on the QSPCs. In 
terms of demographic characteristics, we do assert that there is little impact 
on program performance other than for the younger participants. 

42 



CM 



CM| 



ROLL 



m 



I 



^ 

CM| 



CM| 



B 

53 



CM| 



O 
CM 



PLEFFIC 



O 

m 

r 



00 



00 



I 

4J 

CO 
CO 
0) 



8 

H 



PLRATE 



CM 



ON 

<fr 

f 



eo 



8 






O 
0) 



* 

00 O 
-H 3 

W -o 

M 

4J 

JJrH 

00 

^JO 

S -8 



-d 

g 

>s 





g rft 

CO CO 



P* 







CO 
0) 
CO 



Q) 
JQ 



0) 

g 





CO 

n 

cd 
<D 



(U 
-H 
O 



4H 
0) 
O 

a 
a 

O 
H 

p 



0) 

H 

H 
O 

a 



Hi CM I 



Bivariate Relationships Between Attitudes of Actors and Performance 

Two attitudes were tested for their association with performance measures 
commitment of the staff to serving the economically disadvantaged and commitment 
of the staff to placement as a goal for Title I programs . 

The commitment to the economically disadvantaged is an attitudinal measure 
reflecting the extent to which the management level staff articulated preferences 
for using Title I resources to help primarily economically disadvantaged 
participants (as opposed to other possible target groups). Judgments about 
this attitude were made by our field staff following extensive field work in 
the sites, observations, review of documents 9 and lengthy interviews. The 
attitudes among the 12 planning sites ranged from very strong and explicit 
preferences for serving the disadvantaged to moderate commitment. 

In testing the association of this variable with performance measures, we 
did not attempt to reproduce exactly the research reported earlier (Ripley and 
Associates , 1978:66) between attitudes and characteristics of persons who 
were served, and the results of the present research are not as conclusive as 
those reported earlier. Seven of the planning sites were judged to have very strong 
and explicit commitment to serving the economically disadvantaged with Title I 
resources. Four of those sites also performed best in terms of actually 
serving the economically disadvantaged. (Service to the economically dis- 
advantaged was measured three ways: in terms of the percent of enrollees 
who were classified as economically disadvantaged, the percent of all 
participants placed who were similarly classified, and the proportion of 
economically disadvantaged persons placed relative to the percent and 
proportion of economically disadvantaged enrollees.) In these four sites, 
the expectation about attitudes and performance were confirmed. 

But in two of the seven prime sponsors where the staff had very strong 
commitment to serving the economically disadvantaged the rate of actual service 
to them was lowest of the 12 prime sponsorships: the remaining "very strongly" 
committed prime sponsorships had relatively low performance. The one prime 
sponsorship where commitment to the economically disadvantaged was judged to 
be only moderately strong nonetheless displayed the highest rate of actual 
service to economically disadvantaged of any of the 12 planning prime sponsors. 
There was no relationship apparent between the level of commitment and placement 
rates or cost measures. 

These results suggest that although a strongly held attitude on the part 
of the management level staff for serving the disadvantaged may be important to 
achieving high actual levels of service in some cases, the attitude alone will 
not assure good performance. 

Another attitude examined was the staff's commitment to placement as a goal 
for Title I programs. The commitment to placement is an attitudinal measure 
reflecting the extent to which placement was articulated and regarded as a goal 
for Title I programs by the management level staff of the prime sponsors. 
Judgments were made by our field staff following site visits, and ratings ranged 
from very strong and explicit commitment to placement to limited commitment. 

The analysis of the relationship between this attitudinal variable and 
actual performance showed some relationships were present as expected, although 
not as dramatically strong as were expected. Correlations between the ratings 
on this variable and performance measures (using only on-site quarters) showed 
that there was a weak association between level of commitment to placement and 
overall placement rate (PLRATE r=.22) which suggests that in prime sponsors 
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where commitment was highest , there was a weak tendency for placement rate to 
also be high* There was no association between this commitment measure and 
the indirect placement measure , however . 

There was a reasonably strong inverse relationship between the stronger 
commitments and the rate of nonpositive terminations (r=-42), indicating that 
in prime sponsors with stronger commitment to placements, the nonpositive 
termination rate was distinctly lower . There was no relationship found between 
the commitment to placement and either of the cost measures . 

The relationship between strength of commitment to placement and actual 
placement rates was not strong, but this finding is not illogical. A staff 
commitment will not by itself guarantee placement success, but the presence 
of such a commitment will be likely to facilitate other actions that 
can directly affect placement, for example, monitoring and coordinating job 
development activities of subcontractors, or changing training curricula as 
occupational trends in the labor market area change. 

This investigation between attitudes and performance has not been 
comprehensive, but it has revealed a modest association between attitudes 
and performance in the 12 planning sites* We would conclude from this 
exploration that while the attitudes of actors do not pose unsur mount able 
constraints on performance, neither does the presence of felicitous attitudes 
guarantee good performance. Attitudes can affect performance indirectly, 
and other factors besides attitudes in the 12 planning sites studied offered 
greater explanation of variation in performance . 

Bivariate Relationships Between Previous Program Operations and Performance 

We treated program mix as an indicator suggestive of previous program 
operations. We were less interested in the constraints that previous decisions 
about program mix pose for subsequent years 1 program mix decisions (the so-called 
"incrementalist" argument) than we were in the effects that program mix 
commitments have on program performance. Practitioner wisdom suggests that 
training programs, although they are more costly, result in better placement 
rates than do work experience or PSE. Program mix is a variable that is 
moderately manipulable by manpower staff* 

The best test of the relative effects on program mix decisions on performance 
would be to examine the placement rates and cost measures associated with each 
program category classroom training, OJT, and so on and compare them. 
Unfortunately, such data are not available, and the best substitute available 
is to relate enrollments and expenditures for each program component with the 
prime sponsor f s overall placement rates and cost measures. 

We measured program mix enrollments by dividing the cumulative number of 
participants in classroom training, OJT, work experience, PSE, and other 
activities by the cumulative number of total enrollees for each quarter. 
("Other" was not a reporting category after FY 76; participants in the governor's 
5% vocational education category were not included.) The percent of program 
mix expenditures was computed similarly by dividing the total cumulative accrued 
expenditures into the cumulative expenditures for classroom training, OJT, work 
experience, PSE, services, and other activities. (The expenditures from the 
governor's 5% vocational education funds were not included.) Data came from 
the quarterly reporting forms for all measures of program mix. 
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We examined the 12 prime sponsors in the planning study for all quarters 
except September, 1974, and also for just the end of fiscal year quarters* 
The results of the correlations are summarized in Table 8. 

The relationships in the 12 planning sites between classroom training 
and OJT (using both expenditures and enrollments) and placement rates were 
less evident than expected. Classroom training expenditures and enrollments 
were moderately associated with the indirect placement measure (PLEFFIC) 
(r f s in the .30 to .33 range), but not with the overall placement measure 
(PLRATE) . OJT expenditures were not associated with either measure of 
placement; enrollment in OJT was associated, but only weakly, with the indirect 
placement measure but only for the end of fiscal year quarters (r=.23). Both 
classroom training and OJT expenditures were associated with higher costs 
per enrollee (r=.31 to .51) and with higher costs per placement (r^.24 and 
.33). The proportion of enrollees in classroom training and OJT was strongly 
associated with cost per enrollee (r=.45 to .76) and to a lesser extent was 
associated with higher costs per placement. 

While expenditures and enrollments for classroom training and OJT did 
not show the expected strong associations with the measures of placement, 
the proportion of expenditures on work experience did show a distinct de- 
pressing effect on both measures of placement, but especially on the indirect 
placement measure (r's ranged from -.25 to -.48). Work experience expenditures 
were moderately associated with lower costs per enrollee but not with cost per 
placement. 

The proportion of prime sponsor expenditures for services was weakly 
associated with improving performance on both measures of placement (r's from 
.24 to .29). 

Program mix commitments are fairly manipulable by staff, and based on this 
bivariate analysis we could recommend that prime sponsors seeking to improve 
their placement rates should minimize work experience commitments. Resources 
spent on services will help improve placements, but not dramatically. The 
apparent lack of a positive relationship between commitments for training 
and placement does not support a recommendation that prime sponsors should 
avoid investing in classroom training and OJT, however. We continue to 
believe that placement success and benefits to the participants are enhanced 
by commitments to classroom training and OJT. 

Combined Effects of Context Variables on Performance 

All the relationships discussed so far have been bivariate examining the 
association between pairs of single explanatory factors and single performance 
measures. While this approach can reveal the presence of Important relationships, 
it does not take into account the effects that other variables may introduce. 
Prime sponsor staff, of course, do not operate in an environment in which 
only one factor is impinging on each performance measure there are many 
interactions occurring between a host of potential explanatory factors and 
performance measures. In the preceding section we employed a verbal and 
qualitative multivariate analysis of the impacts that numerous context features 
had on the three models of planning. In this section, we will employ a 
quantitative analysis of the impacts that selected context features exerted 
on performance in the 12 planning sites. The results will provide a descriptive 
map of which factors were influencing program performance in the 12 prime 
sponsorships in the planning study, and will in effect set a context in which 
to understand the additional impacts of planning systems and other variables. 
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Use of multiple regression analysis gives the researcher the ability to 
analyze statistically and simultaneously the effects of several explanatory 
variables on the dependent performance measure. Multiple regression can 
indicate how much variance the combination of explanatory variables account 
for, and it can indicate the relative importance of the explanatory variables* 
While these are important advances over bivariate analysis, multiple regression 
is not the ultimate tool of analysis it has many limitations as well. This 
report is not the vehicle for a treatise on regression analysis, but the 
reader should be aware that the validity and content of the results of 
regression analysis can change often dramatically- depending on a number of 
factors, for example: 1) the total number of observations (cases) entered 
into the regression equation; 2) the choice of specific quarters to analyze 
(only end of fiscal year, all quarters, only FY 78 quarters, and so on); 
3) the identity of specific prime sponsors to analyze; 4) the number and 
identity of the explanatory variables used in the equation; 5) the presence 
of statistical association among the explanatory variables in the equation; 
and 6) the assumption of linearity of relationships imposed by regression 
analysis (as well as correlation) analysis. 

We have tried to balance these and additional considerations in order to 
obtain valid results. We ran regressions for the 12 prime sponsors grouped 
together (limitations on the total number of observations precluded comparing 
prime sponsorships clustered in the three planning models separately). We 
analyzed data both for all quarters except 9/74 (in order to maximize 
observations and increase confidence in the results) and for all end of fiscal 
year quarters alone (in order to make the results of the regression analysis 
consistent with the bivariate analysis) . We selected the explanatory variables 
based on theoretical concerns and previous research findings; we were also 
guided by the results of the bivariate analysis, the presence of interrelationships 
among the independent variables, and the results of other multiple regressions 
not reported here. The resulting list of explanatory factors includes seven 
variables: percent of expenditures for training activities (classroom and OJT 
combined), the percent of expenditures for services, the percent of female 
enrollees, the percent of enrollees lacking a high school education or 
equivalent, the percent of enrollees unemployed at the time of entering CETA, 
the percent of enrollees on welfare, and the quarterly unemployment rate. 

The regression results reported here should be considered as suggestive 
and descriptive for the 12 planning sites. The results are not generalizable 
to other groups of prime sponsors, as comparisons below will illustrate. 

Proportion of Variance Explained. Although regression equations yield a 
plethora of data, we are interested primarily in two questions: what is the 
combined impact of the explanatory variables on a given performance measure? 
(this is discussed in this subsection) , and what is the influence of an 
individual variable relative to the other explanatory variables (this will be 
discussed in the subsection immediately following the present one) . 

Table 9 shows the combined impact that the seven explanatory variables 
had on each of the five dependent performance measures. "Variance explained" 
(denoted by the symbol "R^") i s a statistical concept derived from the procedure 



Table 9i PROPORTION OF VARIANCE EXPLAINED (R 2 ) WHEN SEVEN EXPLANATORY 
FACTORS ARE REGRESSED ON PERFORMANCE MEASURES I/ 





; PLRATE 


PLEFFIC 


NPT 


$ENROLL 


$PLCMT 


R 2 12 Planning Sites, 
All Quarters Except 
September, 1974 


33% 


46% 


12% 


27% 


36% 


R 2 12 Planning Sites, 
End of Fiscal Year 


32% 


36% 


19% 


58% 


32% 


Quarters 













I/ The seven independent variables are: a) quarterly unemployment rate, b) pro- 
"" portion of expenditures for training activities (classroom and OJT combined) , 
c) proportion of expenditures for services, d) percent enrollees unemployed, 
e) percent enrollees on welfare, f) percent enrollees without high school 
degree, and g) percent enrollees female. 



of variables is in explaining change in the performance measures. The smaller 
the value of R 2 the less influence the independent variables exerted over the 
performance measure, indicating that other factors, not in the equation, were 

accounting for change in the dependent measures. 

As Table 9 shows, the same combination of seven independent variables had 
quite different explanatory power over the different performance measures. The 
combination of participant characteristics, program expenditures, and unemploy- 
ment rate was least important in accounting for variance in nonpositive 
termination rate (NPT) in the 12 planning prime sponsors. The seven variables 
accounted for only a small amount of variance in this performance measure (12% 
for all quarters together, and somewhat higher 19% for the end of fiscal year 
quarters together). This result is consistent with the bivariate correlations 
reported earlier, and does not change even when other groups of prime 
sponsors from previous studies were analyzing using the same regression equation. 
This clearly suggests that context variables, at least as measured here, are not 
important in accounting for changes in nonpositive termination rate, and that 
other factors are much more important, for example, the nature and quality of 
intake, assessment, referral, counseling, and coaching. 

The combination of seven context variables accounted for roughly one-third 
of the variance in the two placement measures. (The figure was higher for 
the indirect placement measure for all quarters combined). Although this is 
a higher value than the R 2 figure for NPT, it still is not large enough to 
suggest a dominant or controlling effect by the independent variables over 
the placement measures. Two-thirds of the variation in the placement measures 
is accounted for by factors other than unemployment rate, participant 
characteristics, and expenditures for training and services. The results of the 
regression repeated on other groups of prime sponsors underscored the limited 
explanatory power this cluster of independent variables exercised over the 
placement measures. In all of the other regressions, the variance explained 
was lower than the figures reported for the 12 planning sites in Table 9. 



* One of the byproducts of the regression analysis is a regression equation 
that estimates the best possible straight line to represent the relationships 
among the variables in the analysis. R 2 the percent of variance explained 
is a measure of spread between the actual values of the observations and the 
estimated values predicted by the regression equation. The larger the value 
of R 2 , the smaller the spread between the actual values and the predicted values 
of the regression line, indicating a good fit. The maximum possible value of 
R 2 would be 100%. The smaller the value of R 2 , the larger the spread between 
the actual values of observations and the values predicted by the regression 
equation, and the poorer the fit between the actual values and the predicted 
values. 

Readers interested in greater elaboration of regression analysis may 
consult Blalock, 1972: Chapters 17-19. 
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The seven independent variables combined accounted for roughly one third 
of the variance in the two cost efficiency measures.* The implications of this 
general result again are that there is a large margin of variance unaccounted for, 
meaning that factors other than program mix expenditures, participant 
characteristics and unemployment rate are more important in explaining variation 
in cost per enrollee and cost per placement* This conclusion is strengthened 
when the regressions were repeated for other groups of prime sponsors for which 
data were available. In all of the other regressions, the total variance 
explained in the two cost measures was lower than the figures for the 12 
planning sites. In none of the other regressions did the R.2 figure exceed 
22% for either cost measure or for either sample of quarters. This fact 
underscores the atypical nature of the unusually high R for the 12 planning 
sites for end of year quarters for the cost per enrollee measure. 

Recapitulating the results of the total variance explained for the 12 
planning sites, the combination of seven different measures of context, each 
presumed by practitioner and conventional wisdom to pose significant constraints 
on program performance, displayed only limited impact when regressed on each 
of five measures of performance (with the single exception of cost per enrollee, 
where the impact was greater). The general conclusion is strengthened when 
comparisons are made to other groups of prime sponsors; these comparisons 
show the proportion of total variance in the dependent measures explained to 
be even less. 

The implications of these results for prime sponsor staff are that even 
within a context of unemployment rate, programmatic expenditures for training 
and services, and participant characteristics, there is plenty of latitude for 
the staff to affect program performance; factors other than these features 
of context are more important in explaining changes in performance. 

The Impact of Individual Variables. Given the relatively small proportion 
of total variance explained by the cluster of seven context variables, given the 
fact that the proportion of variance explained was even smaller in comparison 
groups of prime sponsors, and given that the precise interrelationships among 
the independent variables is subject to change as variables are added or deleted, 
we opted not to devote excessive space to describing the relative influence 
and interrelationships of the seven explanatory variables in each regression. 
The most important variables in each regression equation can be identified 
(using standardized regression coefficients (BETA f s) to determine relative 
influence), however, to help sort out the results of many of the bivariate 
associations reported earlier and to indicate which single variable among the 
seven exerted the most influence on the dependent performance measures when the 
effects of all the other explanatory factors were statistically controlled for. 

The regression analysis showed that for both measures of placement the 
percent of enrollees lacking a high school degree was the most important variable 
in the regression equation, and that it exhibited a net negative effect on both 
measures of placement rates. (This result was present for both all quarters and 

* The differences in Table 7 between the R^ for all quarters and the R 2 for 
just end of year quarters were reasonably small for all the performance 
measures except for cost per enrollee, when the R increased from 27% to 58% 
for the end of year quarters alone. There is no apparent reason for such a 
large change changes in other groupings of prime sponsors were very small 
except for a unique statistical association among the data points for the 
12 planning sites for those points in time. The large R figure is not 
typical of the other groupings of prime sponsors examined, all of which 
showed very low R 2 figures on this dependent measure. 
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end of fiscal year quarters*) The strength of this participant characteristics 
is consistent with the relatively strong inverse bivariate relationships reported 
earlier for percent of enrollees without high school degrees and the placement 
measures. The regression results indicate that even in the presence of other 
explanatory factors, each of which exhibited some independent association 
with placement rates, the percent of participants without a high school degree 
is the most important factor affecting the dependent variable . (This ff most 
important" label is relevant only in the context of the total variance explained 
by all the independent factors together. Recall that the total variance 
explained for the placement measures was about 33% the percent of enrollees 
without a high school degree was the most important of seven variables that 
together accounted for roughly one third of the change in the placement rates 
of the 12 planning prime sponsors.) 

The percent of expenditures for training activities was found to be 
unimportant in explaining variation in the placement measures for the 12 
planning sites, even when the effects of the other six variables were controlled 
for. While consistent with the earlier bivariate results that showed only a weak 
or absent relationship between training commitment and placement rates, this 
finding is inconsistent with experience and conventional wisdom that suggests 
a strong relationship exists between the commitment to training and the 
placement rates. When the regression analysis was repeated for other comparison 
groupings of prime sponsors the results did confirm the expected relationships. 
For the 30 prime sponsors not in the planning study, the expected relationships 
between training expenditures and placement rates were present, both in 
bivariate and regression analysis results, and the proportion of expenditures 
for training activities was the most important single variable in the regression 
analysis. 



ttotipositive termination rates, in the 12 planning sites, the unemploy- 
ment rate was the most important single explanatory variable, but the importance 
of this variable must be assessed in the context of the very low total amount of 
variance accounted for by the seven variables together. 

The percent of expenditures for training activities was found to be the 
most important single explanatory variable in the regressions for both measures 
of cost efficiency. 

Conclusions from the Regression Analysis. Regression analysis helps 
enlarge understanding of variance in the performance measures by considering 
more than one explanatory factor at a time. The analysis reported here 
regressed seven independent variables on performance measures. The seven 
independent variables were selected because ttiey posed, at least potentially, 
large constraints on performance, with a concomitant diminished degree of staff 
flexibility and latitude to alter performance. The results of the regression 
analysis lead to the following conclusions: 

1. A low proportion of variance in the dependent performance measures 
is explained by the unemployment rate, program expenditures, and demographic 
characteristics of participants, ^here is a great deal of latitude for prime 
sponsor staff to affect program performance, even after these factors have 
been taken into account. This result is true for the 12 planning sites as 
well as comparison groups of prime sponsors. 

2. Unemployment rate is the single least important explanatory factor 
affecting change in the performance measures among the planning sites. This 
is consistent with the bivariate results discussed earlier and with research 



results reported elsewhere (RIpley and associates, 1978), but it runs 
counter to conventional wisdom . The point that needs to be made to manpower 
staff is that unemployment rate may be having a greater impact on the hearts 
and minds and perceptions of staff than it is actually having on prime sponsor 
program performance . Attitudes about the impact of unemployment may be a much 
greater constraint on performance than the actual unemployment rate* 

3. The demographic characteristics of participants do not control 
performance. The multivariate results underscore and strengthen our earlier 
conclusions derived from bivariate analysis. For only two of the performance 
measures was a demographic characteristic found to be the most important 
explanatory factor* For both of the placement measures, the percent of 
enrollees without a high school diploma exerted dominant influence, in the 
direction of lower placement rates. For the other measures of performance, 
demographic characteristics were not the most important explanatory variables. 

4. The proportion of expenditures for training activities (combining 
classroom and OJT) was not important in accounting for variance in the place- 
ment measures in the 12 planning sites. The percent of training expenditures 
was the most important variable accounting for changes in the variance of both 
measures of cost efficiency. These results were consistent with the bivariate 
relationships among the 12 planning sites, but they were surprisingly different 
from common expectations about the association between higher training commitments 
and better placement rates. Regression analysis using comparison groups of 
prime sponsors upheld the expected relationships, however. 

5. The 12 planning sites are not representative of other groups of 
prime sponsors, and generalizations from the planning sites based on the 
regression analysis must be limited. The regression analysis results accurately 
portray the effects that seven context variables exerted on performance 
measures for the 12 planning sites, sites chosen for analysis because of the 
good reputations of their planning systems, and these sites are not necessarily 
typical of other prime sponsors. In comparisons with other prime sponsor 
groups, the regression analyses were repeated with the same cluster of 
explanatory variables, and they were found to have less potency in 
accounting for variance in the performance measures. Additionally the relative 
influence of individual variables changed. 

C. RELATIONSHIPS BETWEEN MANAGEMENT VARIABLES AND PERFORMANCE 

Unlike the contextual (or external) variables used to explain program 
performance in the previous sections, the concepts in this section, falling 
under the rubric of management decisions, were chosen to reflect qualities of 
and processes within the CETA staff organization itself. In general, we focused 
our attention on three broad categories of management indicators: staff 
characteristics, administrative characteristics, and relations vjith other 
actors. Values for the indicators designed to measure these aspects of 
management decisions were arrived at through 1) the observations of two-person 
teams who spent about two separate weeks in each prime sponsorship, 2) review 
of prime sponsor documents, and 3) extensive interviews with staff, service 
deliverers, advisory council members, and other local manpower actors. 

Among the cluster of staff characteristics we included quality of top 
staff and quality of all staff. Quality of top staff reflects tne professional 
capabilities, experience, and qualifications of the management level staff in 
a prime sponsorship (the director, deputy director, and division or unit heads). 
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It is not limited to just the planning staff. Quality of all staff represents 
similar judgments covering all of the professional staff- For both measures , 
rating categories were very good, good and fair* Both measures are subject 
to manipulation by staff (staff members can be upgraded by hiring practices 
and in service training courses , among other things), 

In the administrative characteristics cluster, we included programmatic 
integration, administrative integration, and characteristics of planning 
systems. Programmatic integration refers to the ability of CETA participants 
to move within Title I program components and to move between CETA titles. 
Ratings of mobility ranged from high to low for this variable, which is highly 
manipulable by staff. Administrative integration reflects the degree to which 
administration of Titles I, II, and VI is handled by separate staff units or 
by the same staff. Ratings ranged from high administrative integration (the 
same staff people oversee all CETA titles) to low (completely separate staff 
units administer the titles). This is a variable that is moderately manipulable 
by the staff in the short run. 

The features of planning systems that we examine included responsibility 
for service delivery, use of RFPs in selecting service providers, quality of 
monitoring, and quality of evaluation. Responsibility for service delivery 
refers to the local arrangements for providing manpower services, and indicates 
whether services are entirely subcontracted, entirely run in-house by prime 
sponsor staff, or whether a mixed mode of delivery is used. This is a variable 
that is moderately manipulable by the staff. 



use of RFPs indicates the extent of the staff's use of requests for 
proposals in the selection of service deliverers, and the formal or informal 
nature of those RFPs. Ratings ranged from high reliance on formal RFPs to no use 
of RFPs. The quality of monitoring refers to the staff f s supervision of 
subcontractors and staff units responsible for service delivery to participants. 
Monitoring techniques range from high (extensive use of quantitative and qualitative 
program reviews based on monitors 1 field visits and desk reviews) to low 
(limited, infrequent program review, based on desk review only). The quality of 
evaluation refers to the staff's assessments of the systemwide performance of 
manpower programs and reflects the nature and extent of such reviews. Ratings 
ranged from high (extensive quantitative and qualitative aspects in widespread 
program evaluations) to low (minimal evaluation, stressing only a few quantitative 
measures). The use of RFPs, the quality of monitoring, and the quality of 
evaluation are all highly manipulable by the manpower staff. 

In the cluster of management indicators called relationships with external 
actors we include the role of the Employment Service as a service provider, 
the involvement of business in manpower programs, and the level of conflict in 
the prime sponsorship. The ES role refers to the degree to which ES provides 
manpower services in the prime sponsorship. Ratings ranged from high to low 
and reflected whether the ES received a substantial proportion of the prime 
sponsor's Title I budget to provide at least two different functions (such as 
OJT and intake services) or whether the ES had no role or only a minimal role 
(for example, verification of placements). The extent to which the staff 
actively solicited and obtained a role for business in manpower decisions and 
programs resulted in ratings that ranged from high, explicit business involvement 
(for example, business was involved in service delivery, or staff committed a 
large portion of funds to OJT, or placements were made in the private sector 
as a result of aggressive staff efforts) to non-involvement of business other 
than nominal membership on the manpower advisory council. The level of conflict 
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Ln the prime sponsorship is a measure of the nature and extensiveness of 
lanpower-related disagreements within the prime sponsorship* This measure 
ilso serves as a rough indicator of the staffs 1 ability to control conflict 
d.th existing conflict resolution strategies. Unfortunately, we do not have 
m alternate measure of conflict resolution capabilities available. The 
ratings range from high to low conflict , reflecting a broad systemwide 
issessment of conflict over manpower issues in general , rather than conflict 
>ver specific single issues or decisions. (This measure of conflict is 
>roader than the measure of conflict used in the previous section of report 
>n planning systems, where conflict assessments were restricted to conflict 
engendered by specific decision areas.) In general, relations with external 
tctors are moderately manipulable by the staff. 

Table 10 presents the relationships between the management variables and 
performance variables (using the on-site quarters only) for the planning sites. 
Additional analyses of these relationships were conducted by subsetting our sites 
iccording to the type of planning system. Subsetting by future orientation 
roved fruitless because of the small number of cases and the extremely small 
mount of variation among them. Subsetting by operations management oriented 
iites and future oriented sites together while excluding the crisis management 
:ite affected the results enough to warrant presentation and discussion of 
:he differences. We did not do great violence to the analysis by failing to 
>ursue the differences between future oriented and operations management 
ites separately, both for the methodological reason cited above and because 
he two groups are considered to be similar in terms of management capabilities, 
he important difference with respect to management capabilities is between the 
risis management model on the one hand and the future oriented models on the 
ther. The crisis management site was characterized by a breakdown in 
management capabilities and control, while in the Operations Management and 
uture oriented systems, management capabilities remained intact and able to 
ope with external pressures. 

In general, the table shows that the management of 12 prime sponsorships 
ombined are more strongly associated with the nonpositive termination rate than 
ere the unemployment rate, program mix expenditures, or participant characteris- 
ics, and that the management variables help to decrease the nonpositive termina- 
ion rate. The management variables showed almost no association with the 
ost measures, and therefore the cost efficiency measures of performance were 
ot included in the table for the sake of brevity. The characteristics of the 
taff were the most strongly associated with the placement measures and the 
onpositive termination rate, followed by the extent of business involvement, 
he quality of monitoring, the degree of program integration, and the quality 
f evaluation. 

Among all 12 planning sites, Table 10 shows that both the quality of top 
taff and the quality of all staff had an identical, positive association with 
he two measures of placement. Quality of top staff and of all staff were only 
eakly associated with increases in the overall placement rate (r's = .26 and 
27) , but these factors were significantly related to the placement efficiency 
easure (both r's .42). These results reinforce our earlier findings 
Rip ley and associates, 1978) and the common sense expectation that the 
ature of the people in the prime sponsor staff does affect the nature of 
rogram performance. Directors have good reason, therefore, to seek to enrich 
taff competencies through use of management courses and in-service training 
ourses for their staff members. 



Table 10: Bivariate Relationships Between Management Variables and 
Performance for the 12 Planning Sites, and the Planning 
Sites Excluding Denver, On-Site Quarters I/ 





II Planning Sites Excluding 
12 Planning Sites 2/ jj Crisis Management Site 2/ 


PLRATE 


PLEFFIC 


NPTRATE PLRATE 




PLEFFIC 


NPTRATE 


Staff Characteristics 


.26 


.42 


-.30 


.27 


.53 


I/ 


Quality Top Staff 


Quality All Staff 


.27 


.42 


3/ 


I/ 


.59 


I/ 


Administrative 
Characteristics 


I/ 


.36 


-.29 I/ 


.47 


I/ 


Programmatic 
Integration 


Adminis trative 
Integration 


I/ 


I/ 


-.43 -.22 


I/ 


-.26 


Operating Re- 
sponsibility 


I/ 


I/ 


37 3/ 


I/ 


I/ 


Use of KFP's 


I/ 


I/ 


I/ 


17 


I/ 


.20 


Quality of 
Monitoring 


I/ 


.35 


-.32 


; I/ 


.64 


I/ 


Quality of 
Evaluation 


.28 


37 


-.27 


37 


.26 


I/ 


Relations with Others 


I/ 


I/ 


-.32 


I/ 


I/ 


I/ 


ES Role 


Business Involvemen 


.31 


.38 


17 


.21 


.36 


I/ 


Level of Conflict 


I/ 


I/ 


I/ 


I/ 


I/ 


-.29 



17 Number of observations is 35 for all 12 sites, and 32 for the subset that 
excludes Denver. 

2/ $ENROLL and $PLGMT were not presented due to the low number of significant 
relationships (five) . 



3/ Indicates the Pearson r correlation coefficient was less than the cutoff of .20. 



Many of the program and planning characteristics in the 12 prime 
sponsors were associated with placement or nonpositive termination performance* 
Programmatic integration (the extent to which participants move within Title I 
components or between titles) was fairly strongly related to improved placement 
efficiency (r = .36) and was somewhat related to better performance on 
nonpositive terminations (r = -.29). The quality of monitoring, an important 
part of the operations management and the future oriented planning systems, 
was fairly strongly related to improved placement efficiency rates (r = .35) 
and to better performance on nonpositive terminations (r = -.32). The quality 
of evaluation, another characteristic of planning systems, was related to 
placement and nonpositive terminations, although not as strongly as the 
preceeding factors (r = .28 for overall placement rate and -.27 for nonpositive 
terminations) . The nature of evaluation was not related to the placement 
efficiency measure, however. These results suggest that the prime sponsorships 
in our study were able to improve placements, particularly indirect placements, 
and to decrease nonpositive termination rates, by exposing participants to a 
variety of CETA services rather than just one activity, and by rigorous 
monitoring of subcontractors. The bivariate correlations did not reveal 
relationships between placement rates, or nonpositive terminations and other 
features of planning systems, including use of RFPs, operating responsibility 
(in-house delivery vs. subcontracted delivery of services) or administrative 
integration of titles. (Administrative integration was fairly strongly 
associated with decreasing nonpositive termination rates, however.) Rather 
than concluding these factors are not important or are not related to 
performance, it is more likely that their effects are indirect, and are being 
masked by other variables. 

Of the indicators of prime sponsor relations with external actors, only 
the level of business involvement was associated with the placement performance. 
In those prime sponsorships where the staff had solicited and obtained an 
active role for the private sector, placement performance tended to be better 
(r f s were .31 and .38). A close working relationship with business can 
contribute to better placement in at least two ways. First, by gaining direct 
access to the private sector labor market through involvement of local 
businessmen, placement performance is enhanced, and second, by using the labor 
market information gleaned from business contacts CETA planners are better 
able to develop training programs that more closely reflect employment 
opportunities in the local labor market. The use of ES as a subcontractor and 
the level of manpower related conflict in the prime sponsorship were not asso- 
ciated with placement performance in the 12 prime sponsors, although there was 
a moderately strong tendency for the prime sponsors that used ES extensively as 
a subcontractor to have better nonpositive termination rates than prime 
sponsors that did not use ES as heavily. 

There were few relationships between any of the management indicators 
(including some planning system characteristics) and performance on the cost 
efficiency measures. The quality of monitoring showed a weak association 
with higher costs per enrollee (r = .27), and in the prime sponsors with high 
business involvement the cost per placement tended to be lower (r = -.30). The 
level of manpower related conflict in the prime sponsorships was surprisingly 
associated with better performance on both of the cost measures, and the 
associations were moderately strong (r's = -.36 and -.30). 

While the above conclusions, derived from analysis of all planning sites, 
generally hold up when the analysis is repeated while excluding the Crisis 
Management site, some noteworthy differences do occur. The relationships 
between staff quality and placement efficiency are enhanced (from about the 
.4 level to between .53 and .59), though the relationship with NPT is reduced 
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variety of CETA services rather than just one activity, and by rigorous 
monitoring of subcontractors. The bivariate correlations did not reveal 
relationships between placement rates, or nonpositive terminations and other 
features of planning systems, including use of RFPs, operating responsibility 
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.4 level to between .53 and .59), though the relationship with NPT is reduced 



to insignificance* A similar pattern holds for the relationship of program 
integration with PLEFFIC and NPT. The relationships of administrative 
integration 5 quality of monitoring, quality of evaluation, and ES role with 
NPT also follow the same pattern. The most striking difference between the 
two samples is in the difference between the quality of monitoring and 
PLEFFIC relationships. The magnitudes of the coefficients increases from .35 
to .64, the largest in Table 10. This dramatically underscores the importance 
of good monitoring. Finally, the relationships business involvement and the 
level of conflict with cost per placement are reduced to insignificance. 

In summary , the differences between the 12 planning sponsorships 
contrasted with only those whose management systems are functioning adequately 
lie primarily in the stronger relationships for the latter group between 
management indicators and the placement measures, especially PLEFFIC, 
and the general decrease in the relationships involving NPT. 

D. RELATIONSHIPS BETWEEN PLANNING SYSTEMS AND PERFORMANCE 
Expectations of Relationships 

In general, assuming that the contexts and implementation factors are 
relatively similar, we would expect that those prime sponsors operating with 
a Crisis Management planning system would demonstrate worse performance 
figures than prime sponsors operating with either an Operations Management 
or a Future Oriented planning system. The net effect of unstable influence 
relationships, unmanaged conflict, a lack of routine in decision making, and 
a dysfunctional feedback system is the generation of poor and inappropriate 
planning decisions. These decisions, in turn, will probably lead to negative 
effects on program performance. For example, in a prime sponsor with unstable 
influence relationships and unmanaged conflict, the staff may have difficulty 
in achieving congruity in planning decisions and flexibility in adapting to 
change when necessary.- The service deliverer decisions may reflect this 
difficulty. When it becomes clear that certain service deliverers should 
be dropped or cut back because they refuse to change their programs to keep 
abreast of the changing job market demands, the staff may be unable to act 
because of the entrenched strength of the existing service deliverers. In 
this situation, the decision may be made to keep them and their services. 
If the situation persists, both lower placement and placement efficiency 
rates can result. 

Serious performance problems would also be expected to result in a Crisis 
Management site because of the lack of a functional feedback system. This may 
make both service deliverer evaluations and target occupation decisions particularl 
difficult. Without an effective monitoring operation, the quality of training may 
decrease without staff detection. Consequently, without corrective action, 
ill-prepared clients will emerge and placement rates and quality of placements 
could be damaged. In addition, without a feedback system that keeps abreast 
of the changing nature of job occupations available in the area, the target 
occupation decision may often be outmoded and inaccurate. Consequently, the 
training programs that are designed on the basis of this identification will 
provide people with skills that may be difficult to market. Thus, placement 
rates and the quality of placements would be predicted to decline. 

In general, we would not predict much difference in the performance 
measures between sites that fall into the Operations Management or the Future 
Oriented planning systems. In these sites, the decision making processes 
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for planning are basically the same. Stable and well-oiled operations with 
functioning feedback systems that do supply important information in a timel 
fashion would be in place. Well thought out planning decisions should resul 
and the performance consequences should be positive. 

However, as the discussion below illustrates, there is some evidence to 
indicate that the future oriented sites performed slightly better, on the wh 
than the operations management sites. We would like to suggest that these 
differences could , in part, be explained by the presence of distinct forms 
of job market feedback mechanisms that appear in the future oriented sites. 
While all details of these mechanisms could not be explored, we did observe 
that the mechanisms focused particular attention on involving the business 
community. Rough indicators show that the future oriented sites did have hi 
and more explicit business involvement than other sites. The form of this 
involvement varied. In Baltimore, formal Labor Market Advisory Councils wer 
established as the institutionalized link with the private sector. In 
Penobscot, the mechanism was more informal. In addition to utilizing job 
developer information, the director keeps abreast of business activity by 
attending Chamber of Commerce and Rotary Club meetings in the area. 

While conclusions from these few examples can only be tentative, we wou 
like to suggest that the examples indicate the usefulness of institutionaliz 
relationships with the business community. Certainly the correlations betwe 
business involvement and placement rates, presented in Table 10, supports th 
This relationship can be utilized not only for job placements but also for 
longer term forecasts of labor demands. (From this perspective, the purpose 
of Private Sector Incentive Program, with its PIC council, makes sense. 
How well it works in practice, of course, remains to be seen and depends, 
at least in part, on the details of implementation in prime sponsorships.) 

Observed Relationships Between Planning Models and Performance 

Examination of performance data for each of the twelve sites and each 
of the three planning models reveals general support for the expectations 
suggested above. In general, Denver, the Crisis Management site, compared 
unfavorably with the other sites and models using standard measures of 
performance. (Data about Denver is only, presented here as being suggestive 
of what might be found in other sites fitting the crisis management descript 
in no sense is it predictive or generalizable since it is only one case.) 
In general, the Future Oriented sites performed somewhat better than the 
Operations Management sites but the difference is not great, and if the sit 
examined individually (Tables 11 and 12) , a good deal of variation among sit 
emerges . 

Table 11 shows, for example, that although the Future Oriented sites, o 
average, performed better than the Operations Management sites across all fi 
there are Operations Management sites that performed equally well (for examp 
San Francisco and Bergen County). Additionally, there is at least one Opera 
Management: site that more closely resembled Denver that the other Operations 
ment sites (i.e. Omaha) along the selected standards of performance. 

If each of the sites is examined for each of the quarters in which 
the study took place (12/77-6/78) , much the same pattern is present (Table 1 
There is little difference between the Future Oriented and Operations Manage 
ment sites, and there is a rather extensive variation within the models, 
especially within the Operations Management model. In general, the 
Future Oriented and Operations Management sites (with the notable 
exception of Omaha) show a pattern of improved performance on the 



Table 11: INDICATORS OF PERFORMANCE FOR 
PLANNING PRIME SPONSORSHIPS, JUNE 1978 



Prime Sponsor 


PLRATE 


PLEFFIC 


NPT 


$ENR 


$PLCMT 


Future Oriented 












Baltimore 


46 


20 


14 


$633 


$1378 


Penobscot 


31 


23 


21 


$997 


$3266 


Syracuse 


43 


21 


14 


$1306 


$3059 


FO Average 


40 


21 


16 


$979 


$2568 


Operations Management 












San Francisco 


43 


25 


20 


$1431 


$3332 


Bergen County 


39 


26 


21 


$1353 


$3476 


Gulf Coast 


35 


6 


7 


?1346 


$3884 


Heartland 


41 


6 


24 


$779 


$1907 


Atlanta 


36 


22 


37 


$1053 


$2960 


Omaha 


25 


19 


30 


I/ 


I/ 


Albuquerque 


34 


18 


27 


$953 


$2775 


King-Snohomish 


28 


23 


18 


$1675 


$6027 


OM Average 


35 


18 


23 


$1227 


$3480 


Crisis Management 












Denver 


23 


17 


43 


$642 


$2808 


U.S. Average 


28 


15 


29= 


$1091 


$3940 



!/ Correct data not available. 
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efficiency (PLEFFXC) , while Denver shows an erratic and generally declining 
pattern of performance. 

Table 13 compares the Future Oriented and Operations Management model 
averages for three standard indicators of performance during FY 78 f and shows 
a statistically significant difference between the models on placement rate, 
placement efficiency, and non-positive termination rate (i.e-, if other sites 
were studied and placed in the models, there is better than a random chance 
that similar differences in the performance indicators would be present.) 
Denver is not compared statistically to the other two models because it is 
impossible to argue that it is statistically representative of other Crisis 
Management sites (mean performance indicators are simply presented to provide 
rough comparison with other models). 

One of the features differentiating the Future Oriented sites from the 
Operations Management and Crisis sites is a deliberate long-range programmatic 
orientation. Future Oriented sites make program decisions based in part on 
how they want their manpower delivery system to perform several years from the 
time the decisions are made. Table 14 presents some data on change in three 
performance measures, change that is at least partially attributable to differences 
in the planning models* The Future Oriented sites show the most improvement 
in placement rate and in placement efficiency, while Operations Management 
sites improve to a lesser extent, and the one Crisis Management site shows 
a decreasing performance on all three indicators. Although the Operations 
Management sites show more substantial improvement on non-positive terminations 
(NPT) , the Future Oriented sites performed at a higher absolute level in 
6/78, which cotald possibly be an optimal level of performance. By 6/78, the 
Future Oriented sites were showing the msot positive level of performance on 
all three indicators, and Denver was showing the most negative level (both in 
absolute level of performance and in the amount of change from 6/76) . 

The results of planned versus actual comparisons on selected measures for 
the 6/78 quarter are shown in Table 15. They substantiate the expectation that 
program performance in a Crisis Management site will not be as good as perfor- 
mance in either the Operations Management sites or the Future Oriented sites. 
The data do not reveal any systematic difference in the performance of the 
Operations Management sites as compared to the Future Oriented sites. 
Generally, both the Future Oriented and Operations Management sites came very 
close to meeting their planned goals of enrollments, expenditures, placement 
rates (PLRATE) , placement efficiency (PLEFFIC) , and non-positive terminations 
(NPT) varying from the planned figures by no more than a few percentage 
points. Denver, on the other hand, varied from its planned rates of achievement 
by as much as 26% (enrollments) , and varied from at least three other of its 
goals by a minimum of 17%. 

Summary . As expected, the evidence discussed above indicated that the 
Crisis Management site had poorer performance scores than either the Opera- 
tions Management sites or the Future Oriented sites, regardless of how performance 
was measured or analyzed. Using the standard indicators of performance, the Crisis 
site generally showed the worst performance of any of the twelve sites, moving 
from reasonably good performance in FY 76 to unsatisfactory levels of performance, 
and showing extensive deviation from its planned performance levels. 

Also, as expected, there was little difference between the performance 
levels of the Operations Management model and the Future Oriented model. 
Taken separately, the sites exhibited wide-ranging performance levels with 
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Table 13s A Comparison of FY 78 Performance for Planning Sites, by Model 



Sites 


PLBATE 


PLEFFIC 


NPT 


Future Oriented (N=3) 


31.31, 


17% -j 


15%^ 




r i/ 


I/ 


\- 


Operations Management 


28.5%^ 


15%-' 


227S 


Crisis Management 


26% 


20% 


36% 


(N=l) 









I/ Using a one-tailed T-test, this difference is statistically significant 
~" at .0005 level. 

2/ This difference is statistically significant at a .005 level. 
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some Operations Management sites looking more like Future Oriented sites than 
the rest of the Operations Management sites , and with other Operations. Manage- 
ment sites looking more like what is expected from a Crisis Management site* 
Taking each model as a whole, small, and sometimes significant, differences 
were found on the selected measures of performance with the Future Oriented 
model usually showing a stronger level of performance when compared with the 
Operations Management model* The Future Oriented model also seemed to be 
most able to effectuate substantial change in performance by surpassing the 
rates achieved by the Operations Management model. Sites using both models 
were clearly able to perform at about planned levels. 



E. CONCLUSIONS 
Summary 

In the preceding sections, we have systematically investigated the 
relationships between indicators of local context, management characteristics, 
and planning with indicators of program performance. The diagram below 
summarizes the general relationships that we have found in the 12 planning 
sites. The broken lines indicate that the relationships were only weak or 
nonexistent; the solid lines indicate that a definite relationship was 
present. (These relationships are consistent with previous findings reported 
in Ripley and Associates, 1978.) 
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As in Section II, where we saw that the staff was able to control the 
nature of their planning systems by conscious manipulation of selected elements 
of their management environment, the most striking theme emerging from the 
analysis of program performance is the degree of control that prime sponsor 
staff can exert over their program performance. If we classify the indicators 
of context and management characteristics in terms of the extent to which the 
staff can directly control or shape them, a very interesting pattern emerges. 
The explanatory factors that are more manipulable by staff are the ones that 
tended to be most highly associated with performance variations in a positive 
way, whereas the explanatory factors over which the staff can exert less 
direct control were not strongly associated with performance variations. 
This evidence suggests strongly that prime sponsors seeking to improve their 
program performance can do so by focusing and directing attention on manipulable 
elements of their local environments, especially on elements of management. 
Program performance can be improved in this manner. Improvements in program 
performance are not much constrained by less manipulable elements of context 
such as economic conditions and demographic characteristics, despite the 
contrary assertions of conventional wisdom. 



The factors that we found to be most strongly associated with better 
performance were the quality of the staff, the nature and extent of business 
involvement , and the quality of monitoring. All of the factors are highly 
manipulable by staff. We would encourage any prime sponsorship seriously 
interested in improving program performance to take steps in these areas. 

Quality of staff can be improved through such measures as engaging in 
very deliberate and careful hiring decisions; providing in-house training 
(available from CETA consultants and DOL seminars, for example); exposing 
key professional staff to all parts of the local manpower delivery system 1 s 
administration (thus fostering diversity in staff experience); temporarily 
exchanging staff with other prime sponsors or with local delivery agents; and 
granting staff time off for formal education leading to professional development. 

Quality of monitoring can be strengthened in a number of ways, including: 
clear assignment of a staff person to each subcontractor (to give the deliverer 
a direct liaison to the staff); regular visits to subcontractors (twice a year 
at least); frequent (daily or weekly) telephone contact between the staff 
monitor and the subcontractor (keeping the lines of communication open) ; 
inclusion of other staff, especially planners, and advisory council members 
on monitoring visits to subcontractors; regular, routine feedback sessions where 
the data gathered through monitoring is shared with the service deliverers; 
and implementation of a review mechanism to assure that corrective actions 
have been satisfactorily made. A strong data base is essential to good 
monitoring and making intelligent followup decisions that are credible both 
on programmatic and political grounds. 

The involvement of the business sector in prime sponsors' manpower programs 
is of course one of the aims of the new Title VII Private Sector Initiative 
Program. While the role of business was not a concentrated focus of the present 
research, we do have good evidence that such involvement can result in good 
program performance. Some of the ways prime sponsors can strengthen ties with 
business are by developing systematic contacts with business, of both a formal 
and informal nature; keeping abreast of local employers 1 plans for hiring, 
firing, occupational needs, and their projections of labor market changes; 
maintaining an active and energetic public relations campaign with the business 
community to make business managers and personnel managers aware of CETA and 
to encourage them to hire CETA graduates through the use of slides, movies, 
speakers, and political officials. 

One concrete way of involving business (which does not depend on PSIP) 
is for a prime sponsorship to ask certain businesses to review their training 
programs, to assess their procedures for placing participants in the private 
sector, and to tell them systematically about their experiences (including problems) 
with OJT and other CETA programs. This charge would be clear and specific and 
would ask business representatives for advice in areas they presumably know 
best. A considerable amount of general intelligence of value to the prime 
sponsorship staff could be gleaned from developing this kind of informational 
relationship . 



IV. EXEMPLARY APPROACHES TO CRITICAL ELEMENTS OF PLANNING AND MANAGEMENT 



This section moves away from a focus on general patterns found by analysis 
to a focus on concrete examples of different productive approaches to planning 
and management elements* We intend this to give additional substance to our 
discussion. We have drawn not only on examples from the 12 planning sites but 
also on selected examples from other prime sponsorships in which we have done 
field work in recent years. This discussion is not intended to suggest that 
there is only one "correct" way to undertake any particular feature of prime 
sponsorship management and planning. However, we feel that presenting brief 
descriptions of approaches that have worked well in specific instances will 
help generate ideas in other prime sponsorships about alterations that might be 
worth trying. 

Seven elements of planning and management will be addressed: 

1. Manpower Planning Councils 

2. Monitoring and Evaluation 

3. Universe of Need and Target Groups 

4. Intake and Assessment Strategies 

5. Labor Market Analysis 

6. System Design: Service Deliverer Selection, Service Delivery Systems, 
and Performance Contracting 

7 . Participant Placement Strategies 

In the discussion of each of the seven areas we will use a similar format. 
The key issue concerning the area will be defined and the advantages and dis- 
advantages of the approaches will be outlined. We will consider the critical 
tradeoffs in pursuing any alternative that we suggest. The general introduction 
will be followed by a brief description of approaches employed by one or more 
prime sponsors. 

We do not view this as a technical assistance guide. Such guides are 
already available and prime sponsors would be well advised to read them. 
But, detailed "how to" guides tend to suffer from a common limitation: they 
assume that prime sponsors primarily face technical problems that can be 
solved with the application of knowledge. Our research on prime sponsors 
indicates that the availability of technical knowledge and competence, while 
important, is only one factor that explains the prime sponsor's approach and 
degree of success with CETA program planning and management. Consequently, 
our discussions attempt to take into account the impact of the CETA prime 
sponsor's environment with its range of organizational and political forces 
that impinge on decisions. Our central purpose is to help clatify the choices 
that prime sponsors have and to demonstrate the utility of various approaches 
that have been successfully implemented in specific cases. 



A, MANPOWER PLANNING COUNCILS 

CETA legislation and regulations suggest broad and ambiguous roles for 
manpower planning councils (MFCs) . They are supposed to assist prime sponsors in 
planning, evaluation, and identification of job opportunities. Groups that 
should be included on councils are listed- It is noted that MFCs should be 
advisory to the chief elected officials in the prime sponsorship . 

Given this broad mandate, it comes as no surprise that the role and 
impact of MFCs varies widely throughout the CETA system. Indeed the involve- 
ment of MFCs ranges from near total control of the CETA programs in a few 
prime sponsorships to a mere "on paper" existence in others. Membership, 
structure, frequency of meetings, and functions performed are also quite 
variable. 

Under certain conditions, MFCs can be very effective manpower planning 
and evaluation tools, which are most helpful to prime sponsor staffs. Many 
prime sponsors that initially minimized MFC participation have come to realize 
their potential substantive and political utility. This section outlines some 
of the advantages and disadvantages of MFCs and the conditions under which they 
will be most useful. 

We have observed MFCs functioning as useful and productive organizations in 
the following substantive areas: 

. selection of service deliverers , including the review of RFPs 
and the selection of PSE projects; 

. the analysis of Labor Market conditions for selection of train- 
ing components (described in more detail in the section on labor 
market analysis below); and, 

. monitoring and evaluating operational programs. 

MFCs are generally not as useful in some other areas of CETA program planning 
and management, but may be in some individual circumstances. 

Several valuable functions that may be performed by MFCs provide further 
justifications for using them. MFCs can be helpful by: 

. providing additional information on factual questions and alterna- 
tive perspectives on how to design and assess manpower programs; 

providing feedback from outside the CETA system on how the program 
is perceived and interpreting the program to various constituencies; 

. legitimizing the decisions of the staff and elected officials and 
acting as a political buffer when particularly difficult choices 
must be made, and; 

increasing public confidence in CETA programs by insuring 
accountability . 

Naturally, these reasons, which are often cited for using MFCs, are seldom 
disputed. However, there are often instances when planning councils tend to 
work in the opposite and negative direction. Among the problems that we have 
observed with some planning councils are the following: 



MFC meetings can be occasions for self-serving log rolls by the 
members to make sure that they get a "piece of the action/ 1 

MFC members can become so involved in the details of the prime 
sponsor f s activities that they unnecessarily interfere in what 
should be basic management decisions such as personnel and 
office reporting responsibilities. 

A great deal of time may be wasted on inane debates over trivial 
matters. 

High turnover of MFC members can lead to constant "orientations 11 
of new members and revisiting overworked questions by new members. 

How can MFCs be made effective? Under what conditions will they provide 
productive adjuncts to the overall planning and management capacities of the 
staff? The basic answer is that the staff must get and keep the right members 
on the MFCs, structure the council effectively to take advantage of their 
skills and limited time, know what decisions they can really help with, and 
provide the information necessary for making those choices. 

Membership of MFCs; Selection, Retention, Proper Use 

Getting the right people on MFCs and keeping them on is the most Important 
condition that must be met if the councils are to be helpful rather than harmful 
to the prime sponsorship. One generally knows who the "right" people are: 
concerned, dispassionate individuals who represent major public and private 
interests within the prime sponsorship. But how does one get these people to 
serve on the MFC? In order to get solid people on the MFC and to retain them on 
it a few conditions must be met. 

First the MFCs must be given substantial responsibilities that are clear 
cut and meaningful. Busy people (usually the most desirable members) will not 
sit still for trivial or perfunctory discussions at least not on a volunteer 
basis. Staff directors frequently complain that their council 1 s membership 
"just isn't interested in CETA programs." This may be true because the members 
see no self-interest in it, or because prime sponsor staff has made the MFC so 
insignificant that only those with generally nothing better to do will continue 
to participate. In other words an uninterested and volatile membership may be 
caused by the prime sponsor staff's poor handling of the council. 

Second, council members are selected to represent a variety of political 
and substantive interests (such as those of education, business, labor, and 
minorities). The specific person that is chosen to represent each of those 
interests should be chosen with utmost care. They should be selected according 
to their reputation for consistent participation in other groups and for their 
willingness to delve into the council's tasks. Not every member of the council 
has to be a perfect participant, but a critical mass of members who will 
shoulder the collective burden of council activities is essential. In selecting 
members for the council, the chief elected officials can play an important role 
by providing information about the general competence of community and civic 
leaders and by giving visibility to their appointment and thus creating an incentive 
for the best potential members to agree to serve. 
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Third, It is often unwise to put major service deliverers on the council 
as voting members . A couple of points in this statement deserve emphasis* We 
say "major" deliverers because we do not mean to rule out all recipients of 
CETA services and dollars: in some communities such a stringent rule would 
eliminate practically anyone who the staff would want to involve. But deliverers 
who have large contracts to run programs should be prohibited both from voting on 
all contract decisions and from joining into discussions of such decisions. 

The involvement of service deliverers at some stage of the council 1 s 
deliberations is important. Perhaps a subcouncil of operators should be 
established to work on the day-to-day problems of coordination and service 
delivery; but their direct involvement in selection and evaluation decisions will 
all but insure that the real decisions will not be made in the MFC or should 
not be made there. In fact, the membership of service deliverers on the MFC 
and the subsequent and predictable self-interested strategies that they employ 
is frequently the justification for not using the MFC in decision-making. Our 
suggestion is to change the membership of the council rather than ignore a poten- 
tially useful part of the manpower system. 

Council Structure 

The structure of MFCs is important because if done properly it will help 
improve the retention of good MFC members and will take advantage of their 
areas of expertise in a more productive way than if they all meet together 
every session. 

MFCs should be subdivided into working subgroups that allow for the division 
of labor. While the names can vary and to a certain extent the functions will 
overlap, we have found that most councils should have subgroups working on 
the following identifiable tasks. 

a. Labor market analysis and private sector involvement (this 
could be a FIG now) 

b. Monitoring and evaluation of ail CETA programs 

c. Selection of service deliverers 

d. Selection of PSE projects and/or job "slots." 

Each subgroup of the full council should be composed of people who bring 
genuine expertise to the group. They should be aided by appropriate staffers 
from the prime sponsorship. The real work of the MFC should take place in 
these committees and the general norm of the full council, which should meet 
less frequently, should be to defer to the recommendations of the subgroups. 
The full council f s business should be to review the work of the subgroups to 
consider matters of general policy. 

Staff Support for MFCs 

If the MFCs are to be useful, they must be supported by the prime sponsor 
staff. The MFCs principle need from the staff is information provided in a 
timely manner. Even the most committed volunteer member needs information 
from the staff. The better members will be inquisitive and request additional 
Information or may suggest deficiencies in the information provided, but they also 
expect the staff to provide baseline information. 
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It is not our view that the MFC should have a separate staff working 
exclusively for them* Rather, the MFC should have access to appropriate staff 
members for the specific tasks that they have been assigned. There should 
be one senior person on the CETA staff that handles the general direction of 
MFC input, but separate staff is not necessary and could well produce new 
problems that are unnecessary and unwarranted. 

In short, MFCs can perform several important tasks and functions for 
CETA planning and management if the several conditions mentioned above are 
fulfilled. MFCs must be composed of the right members and those members must 
stay on the council, they must be structured effectively to take advantage 
of the member's expertise, and the staff support for the council deliberations 
must be provided. Councils should not be run by the staff nor run the staff, 
but they do have a useful role to play in the CETA system. 

Some Examples 

Three excellent examples of planning council participation will be described 
below: San Francisco, Atlanta, and Fenobscot Consortium. Baltimore's creative 
use of Labor Market advisory councils is discussed in the Labor Market Analysis 
section of the report. 

San Francisco, California. The San Francisco CETA program is administered 
by the Mayor's Office of Employment and Training. Decision-making patterns in 
the San Francisco CETA program reflect the local norm in city government that 
encourages public participation. Neighborhood based agencies in the city are 
highly politically sophisticated. The base Title I allocation is about $8 
million. 

The Manpower Planning Council (MFC) members that are appointed by the 
Mayor represent numerous constituencies ethnic, racial, and geographic.* 
While all MFC members have not been equally dedicated, the council as a whole 
can be characterized as aggressive and knowledgeable about CETA. The council 
is chaired officially by the Mayor, but the alternate chairman who always 
presides as a senior member of the Mayor's staff. The CETA planning staff 
provides support for the council. 

The San Francisco MFC vigorously reacts to, debates, and initiates 
planning recommendations and decisions in several areas of responsibility: 
program design, evaluation of RFF proposals, and evaluation of subcontractor 
performance. 

The program design details into which the council has major input include: 
1) the percentage of funds allocated to each Title I program activity, such 
as classroom training, OJT, and so on; 2) the identification of special target 
groups and the percentage service goals for each group; and 3) the establishment 
of overall performance standards for the manpower delivery system, including 
number served, placement rates, and cost per placement. 

The MFC also reviews and evaluates responses from potential contractors 
to staff requests for proposals. They play a major role in selecting the 
subcontractors whose mix of proposed services, budget, and target groups will 
fit the specifications of the overall program design. 



Note that this discussion focuses on the San Francisco MFC before a very 
recent reconstitution of it. 



The MPC f s most important task is its evaluation of subcontractors. The 
semi-annual evaluations are connected with the evaluation of RFP responses, 
Staff monitors who are assigned to each subcontractor write operations 
overviews that incorporate their personal observations, MIS reports 5 program 
plans, and other program data. The monitor also completes a standard 
Monitor f s Review form that details program activities. Finally, each subcontrac- 
tor prepares a self -evaluation report. All of these documents discussing the 
programmatic, fiscal, and administrative activities of the subcontractor are 
submitted to the MFC. The council holds public hearings on program performance 
and identifies problems in subcontractor performance for staff follow-up. 
The results of the evaluations are used extensively in refunding decisions. 

Atlanta, Georgia. The Atlanta CETA program is administratively located 
in the city f s Department of Community and Human Development, but the Director 
of CETA reports directly to the Mayor. An outstanding feature of the prime 
sponsorship, whose Title I program is now about $5 million, is its openness 
in CETA decision-making. The primary vehicle to achieve this is the 35-member 
Employment and Training Advisory Council (ETAC) . Twenty-nine members of the 
ETAC are appointed by the Mayor upon recommendation from the staff and with 
the concurrence of the city council. The remaining six members are appointed 
directly by the city council with the approval of the Mayor. The only city 
employee that may serve as an ETAC member is the Director of the Bureau of 
Personnel. New members are given a one-day orientation by CETA staff. 

The ETAC considers and influences decisions regarding the selection of 
target groups and occupations, program mix, and, most importantly, service 
deliverers. The ETAC is very active and its by-laws provide for dismissal 
from it if a member fails to attend two-thirds of the monthly meetings over 
a six-month period. A one-week notice and an agenda is required before an 
ETAC meeting can be held. Notices of meetings are published in local 
newspapers. 

Since virtually all major Title I functions are subcontracted in Atlanta 
the process of annual selection of service deliverers is a most important one. 
It begins when the staff distributes Requests for Quotations for desired 
services. The RFQS specifying detailed program requirements are widely distri- 
buted to a bidder *s list and advertised in major newspapers. 

Funding recommendations are prepared in subcommittees of the ETAC, which 
are constituted to ensure freedom from conflict of interest. Using formal 
ratings, the subcommittees award each proposal points in areas of demonstrated 
effectiveness (such as placement success), professional qualifications, planned 
performance standards, cost-effectiveness, and supportive services and facilities. 
The individual proposals are ranked in comparison with one another and submitted, 
along with a recommendation, to the full council. Subcommittee rankings are 
generally accepted by the full body, and reconciled into an overall package 
during a meeting with the staff. The staff departs from the session with the 
MFC with a priority designation on each proposal being either a "must fund," 
a "fund if possible," or a "don f t fund." 

There seems to be a genuine give-and-take between the council and the 
staff. While the staff has ample opportunity to try to sway the decisions 
of the council, they tend to limit their influence to those issues directly 
related to the performance and survival of the system as they view it. 
Not the least among their motives is the recognition that the ETAC is a 
valuable buffer between them and disgruntled bidders, but would soon cease 
playing that role if it began to be perceived as a rubber stamp manipulated 
by the staff. 
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Penobscot "Consortium, 'Maine. The Penobscot Consortium Training and 
Employment Administration operates under the jurisdiction of a Consortium 
executive board , which is composed of the six county commissioners from Penobscot 
and Hancock Counties. Not nearly as large as the first two prime sponsors 
discussed above , the consortium 1 s Title I allocation is slightly over $1 million* 

There are two advisory councils 2 one serves Penobscot County ; the other 
serves Hancock County* Members come from a broad range of backgrounds 
ES, vocational schools , community colleges and universities, social service 
agencies , community based organizations, business, labor, and participants. 
Membership has been stable and both the staff and the council agree that the 
council is knowledgeable about CETA programs. Some council members represent 
agencies that have submitted proposals for PSE positions, but the council is 
not dominated by groups that are singularly interested in securing CETA funds 
for their agency or group. 

While the County Commissioners appoint the council chairs, and often 
attend council meetings, they rarely participate actively, and in no way 
dominate council proceedings* The entire council meets monthly and sub- 
committees at least one additional time each month* The councils' three 
subcommittees handle Title VI project proposals, new programs, and monitoring 
and evaluation. Its most important role has been in Title VI project selection. 

An ideal opportunity for the staff to foster citizen participation came 
in the form of the Title VI project funding decisions because the staff recog- 
nized the great potential for conflict and outside pressure. Agencies 
throughout the consortium area were invited to submit project proposals for Title 
VI funding* These proposals were collected by the staff and, after a very 
brief review, passed along to the subcommittees of the advisory councils* 
Subcommittee members spent long hours formulating project funding recommendations. 
The subcommittees established their own funding criteria and evaluated the 
proposals themselves. They also held public hearings so that project advocates 
could offer further support for their proposals. During this process, staff 
support was available, but the councils and the subcommittees were the key 
actors. The final council recommendations were well received by the staff 
and the county commissioners and faithfully implemented. 

B. MONITORING AND EVALUATION 

Monitoring and evaluation are important components of management and 
planning in a CETA prime sponsorship a statement with which few would disagree 
in the abstract. Problems enter in when prime sponsors debate how much resources 
should be devoted to monitoring and evaluation and the purposes that should be 
accomplished with the monitoring and evaluation effort. This section of the 
report will discuss the use and misuse of monitoring and evaluation and how to 
mount relatively simple and inexpensive approaches that are nonetheless quite 
effective. 

Monitoring and evaluation are related, but different, enterprises. 
Monitoring focuses on operational questions keeping track of the system. It 
should be "present" oriented and provide feedback immediately into the 
system. Evaluation, on the other hand, examines the impact of a particular 
program component. Such questions as placement rates, cost-per-placement, 
wages obtained and length of employment of those placed form the heart of 
evaluations. Almost all prime sponsors conduct some sort of monitoring. 
Only a minority undertake evaluations. 
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It is important to point out that in order to conduct effective monitoring 
and evaluation a reliable management information system must be established* 
Without information on the performance of the programs that has been systema- 
tically collected by the prime sponsor staff, it is virtually impossible to 
conduct useful evaluations. The MIS has to be designed not only to fulfill 
minimum federal reporting responsibilities 9 but also to provide information that 
will allow senior staff to answer the kinds of evaluative questions in which 
they are interested. A thorough discussion of MIS is beyond the scope of 
this report, but it is essential to design an MIS with questions of monitoring 
and evaluation _ in mind. 

The Use and Misuse of Monitoring and Evaluation 

A fundamental distinction that must be made in evaluation is whether 
a program is already in operation or whether consideration is being given to 
starting a program. The analysis required for making good decisions about 
which program to operate are very different from the requirements for evaluating 
programs that are already operational. We will focus on evaluating programs 
that are already in operation. In such cases the interest is more upon the 
improvement of services rather than upon evaluating whether or not a service 
is worth keeping. In other words, most prime sponsors are probably going to 
continue to operate vocational training, work experience, public service 
employment, and OJT programs. The key questions are "How can our programs 
be made better? 11 "How much money should we spend on each one?" and, "Which 
agency can best deliver the service?" 

In order to find out how to improve programs the prime sponsor staff must 
monitor them on a regular systematic basis, and then pull together that 
information to determine corrective action and technical assistance. 

Monitoring and evaluation can serve several functions: 

1. provide regular feedback on the operation of programs, agencies, 
and personnel. 

2. determine the nature of corrective action to improve programs, including 
termination if necessary, but, more typically, changes in their design, and/or 
the approach of personnel. 

3. help clarify and re-examine the objectives and underlying assumptions 
of the programs. 

4. develop a more critical attitude among prime sponsor and subcontracting 
agency personnel, which should include opportunities for them to suggest ways 

of improving services . 

5. increase staff morale and commitment as a consequence of the attempts 
to improve the program. 

There are also a number of what Edward Suchman (1970) has labeled as 
"pseudo-evaluations . ff We have observed instances of Suchman 1 s five categories 
of such specious enterprises: 

1. Eye-wash an attempt to justify a weak or bad program by 
deliberately selecting for evaluation only those aspects 
that "look good 11 on the surface. Appearance replaces reality. 
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2. White-wash. an attempt to cover up program failure or 
errors by avoiding any objective appraisal. Vindica- 
tion replaces verification. 

3. Submarine an attempt to "torpedo" or destroy a program 
regardless of its effectiveness. Politics replaces 
science. 

4. Posture an attempt to use evaluation as a "gesture" of 
objectivity or professionalism. Ritual replaces research. 

5. Postponement an attempt to delay needed action by 
pretending to seek the "facts." Research replaces 
service. 

Doing Monitoring and Evaluations How Difficult Are They To Do? 

Perhaps more troublesome than any of the "pseudo-evaluations" is the 
widespread absence of evaluations . The reluctance to get into the eval- 
uation "business" has its roots in a number of apprehensions and misconcep- 
tions about program evaluation. 

One frequently heard lament is "we can't do evaluations, because no 
one here has statistical training and even if some did the rest of us couldn't 
understand them." In most cases sophisticated evaluations, with elaborate 
quantitative manipulations and methodology, are unnecessary. Most prime 
sponsors need practical program evaluations that will help improve the 
delivery of services and the basis for choosing among alternative program 
mixes and service deliverers. In such evaluations, the most difficult calcula- 
tions are no more complicated than balancing a checkbook and can be done on a 
desk calculator. Evaluations can be conducted by staff with little or no 
training in evaluation research if some straightforward principles of logic 
are followed. 

The examples cited below demonstrate a range of approaches to monitoring 
and evaluation from the simple to the complex. Some prime sponsors look at 
a few performance measures and conduct qualitative reviews of subcontractor 
performance. Others collect and analyze mountains of information on the 
economic and non-economic impacts of CETA programs on participants. The 
extent of monitoring and evaluation conducted by any given prime sponsorship 
should hinge on the general availability of staff and resources to conduct 
them. Low cost evaluations that yield very useful information have been 
devised and successfully employed. Moreover, as noted in the discussion of 
manpower planning councils, it is possible to make use of volunteer help in 
conducting monitoring visits and evaluations. The nature of the evaluation 
effort should also be linked to the size of the program. Smaller prime 
sponsors may not be able to afford (or justify) large scale evaluations and 
should lean toward the simplest and least expensive approaches available. 
Larger prime sponsors may be able to afford more complex and systematic eval- 
uations and the stakes of their programs justify larger expenditures of staff 
time and resources . 

Some Examples 

The examples in the section below discuss some useful approaches to 
evaluation that have been conducted in the prime sponsorships we visited. 
We hope that they will help deflate the myth that evaluations are too 
difficult, too costly, or a waste of time. In fact, we think they clearly 
demonstrate the utility of systematic attempts to find out "what works." 



Bergen County, New Jersey, The Bergen County CETA program is operated 
by the Community Action Program. Most of the $5.5 million Title I program, 
with the exception of training courses, is operated by the agency . Although 
Bergen County does not have a sophisticated or heavily staffed monitoring 
and evaluation unit, they nevertheless successfully employ a number of useful 
approaches to finding out what works and how to improve it. 

Most of the monitoring is done by the planning unit f s senior planner for 
monitoring and evaluation (M&E) and his two assistants* The overall purpose 
of their monitoring is to assess the performance of the in-house units 
and the subcontractors in relation to the plan. At least once a year 
all components of the CETA program are monitored : the Vocational Center 
(run by a local school board) , Adult Learning Center (run by a community 
college), and OJT, Work Experience, and multi-service centers (run by the CAP). 
As special needs (or problems) arise additional monitoring studies are 
conducted . 

The M&E unit focuses on client satisfaction with employment and training 
services. All CETA participants are mailed a brief questionnaire with a 
standard set of questions on how they got into the program, what they gained 
from it, what happened to them after participation, and so on. Results are 
analyzed and summarized for the senior staff, the operations staff, and the 
staff of the subcontracting agencies. Each report includes recommendations 
for improvements to correct deficiencies uncovered by the poll of enrollees. 

Parts of the review of agency and subcontractor performance consist of 
on-site visitations, interviews with the staff, and desk reviews of agency or 
component performance. Subcontractors are required to submit monthly reports. 
The vocational training program, run by an outside agency, and the OJT program, 
run by CAP, receive the most attention from the monitoring staff. 

The most interesting aspect of the planning unit's monitoring is their 
work on the OJT program. One staff member is assigned solely to this task: 
he completes a detailed analysis of each OJT contractor's performance. 
Each contractor and participant is interviewed by the monitor. On the basis 
of these interviews, a corrective action report is prepared as necessary, 
submitted to the contractor, and followed -up by the same staff member. Criteria 
for assessing OJT contractor performance are the extent to which the employer 
provides the training specified in the contract and hires people that they 
would not have hired otherwise. The monitoring has had concrete results, 
allowing the staff gradually to reduce the length of training for most OJT 
contracts, thus freeing additional resources. 

A different type of monitoring is done by the prime sponsor's advisory 
council, known as the ETAC. Members of the committee are given the results 
of the monitoring reports, but they also make on-site visits to the multi- 
service centers and to the major subcontractors. The ETAC is given credit 
for pushing the staff and the subcontractors to improve performance, parti- 
cularly in the OJT area. ETAC reviews are conducted in advance of decisions 
about the Title I plan. 

Monitoring is also done by the operations unit chief. While the planning 
unit examines achievement of planned levels of performance, the operations 
director is more interested in day to day problems. Consequently, he inves- 
tigates such questions as: Are the multi-service centers properly completing 
the employability development plans? What is the nonpositive termination rate 
for each center? Each counselor? Why? In order to answer these questions, 
the operations director holds regular meetings with the multi-service center 
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directors and their staffs, makes unannounced visits, and most interestingly, 
sends bogus clients through the system who report back to him about their 

treatment at the centers* 

The various approaches to monitoring have an important impact on the CETA 
programs. The staff views their approach to monitoring as "looking for 
trends." There are no formal outcome evaluations, rather, the staff views 
evaluation as the summary of the monitoring efforts. The staff recognizes 
the value of more formal evaluations, but has not conducted any because their 
MIS has never really provided them with the kind of data that would be required 
to perform them. 

The Bergen example illustrates that useful monitoring can be accomplished 
with a limited investment of staff resources, and with limited sophistication 
about evaluation methodology and statistics. The effect of the monitoring 
activities was summarized by a senior staff member: "it has been instrumental 
in building a level of accomplishment. Over time, we have used the reports 
to gain an increasingly objective sense of our program." 

Penobsco t Consortium , Maine Like the Bergen County programs, Penobscot's 
monitoring system is relatively simple. Weekly, monthly, and quarterly reports 
prepared by the MIS unit are used by both the planning and operations units 
to check on client flow, expenditures, target group service, and so on. 
Full-time monitors make visits on a regular basis to training, work experience, 
OJT, and PSE sites. Findings from the visits are compiled by the MIS unit and 
circulated to the senior staff so that necessary changes can be made immediately. 
The Area Operations Director also engages in personal monitoring. Moni- 
toring is viewed as a means of keeping abreast of short term trends in 
the program. Planning staff are interested in the relationship between planning 
projections and performance; operations people want to know how the different 
Offices of Training and Employment Programs are performing and even how the 
individual job developers and counselors are performing. 

Unlike Bergen County, Penobscot conducts evaluations focusing on longer 
term trends and tests basic questions about whether a program or service 
produces the expected results. Short term evaluations may examine questions 
related to the monitoring concerns. For example, last year they examined 
the relationship between enrollment, termination, and placement of various 
demographic groups in order to reach some judgments about the equity of service 
in the program. Another short term study looked at the relationship between the 
applicant pool, those referred, and those enrolled in STIP. This study showed 
that those enrolled were more representative of the applicant pool than 
those referred, suggesting that referrals were made on the basis of incorrect 
assumptions about the kinds of people best suited for the program. This was 
passed on to the operations staff and changes were made. 

The longer term evaluations are based on surveys administered to all 
Title I participants and a sample of PSE enrollees at regular intervals after 
they terminate from the program 30, 60, 90, and 180 days, and 1, 2, 3, and 
4 years. The surveys probe numerous social and economic effects of the 
program. Control groups will be developed from among those who applied and 
were eligible for the program but did not participate. 

The impact of the monitoring activities is substantial. The operations 
staff is anxious to get monitoring reports and use them to identify and 
treat operational problems. The reports also promote continuous and 
constructive exchanges between the planning and operational units of the 



agency . The results of the longer term evaluation studies are just beginning 
to become available, since the system was set up only a year ago* But* the 
senior staff are committed to using the results of the evaluations to make 
changes, even if they are major ones. 

Baltimore Metropolitan Manpower Consortium, Maryland. The Baltimore 
Consortium consists of the City of Baltimore and the five contiguous counties 
in the SMSA. The Mayor's Office of Manpower Resources is the administrative 
arm of the consortium- With a base Title I allocation of over $15 million and 
a total budget of nearly $70 million, it is clear that the agency can afford 
and in fact needs a large evaluation staff. The Research and Evaluation Unit 
alone has 10 employees, but many more people are involved in monitoring and 
evaluation . 

The Baltimore Consortium makes the standard distinction between monitoring 
and evaluation. As with Bergen County and Penobscot consortium, monitoring 
involves short term assessments of the progress of subcontractors toward 
meeting their contractual goals. Evaluation examines the impact of CETA 
services on participants and the community. Monitoring is done by nine program 
analysts who work in the Grants Management unit. They visit the programs to 
which they are assigned at least once a week to check payroll, attendance, 
and classroom activity, keeping informal records and making a monthly report to 
the Director. The monitors play both oversight and technical assistance roles. 
Much of their time is spent on maintaining an efficient and accurate MIS 
because it generates weekly reports on the performance of the agencies. 

The evaluation staff is divided into teams that evaluate short and long- 
range impact of CETA programs. The teams are divided as follows: manpower 
service center evaluations, youth programs, training components, work 
experience and PSE, and follow-up for surveys of participants six months after 
termination. The members of the evaluation staff report directly to the 
Deputy Director and have no operational responsibilities. Each team, which 
is made up of people with program experience and some methodological training, 
establish an agenda in consultation with the program monitors and the senior 
staff. 

Short term evaluations examine the types of clients served by programs, 
service deliverers, and jurisdictions, or the placement rates by similar 
categories. They also analyze relative costs and conduct surveys of various 
kinds, including employer reactions to manpower trainees. Long term 
evaluations include surveys of program completers after six months, for 
example. In short, the evaluation unit undertakes a series of operationally 
oriented and long-term projects that aid the senior staff in making concrete 
decisions, but their work can also help develop knowledge about the efficacy 
of various manpower approaches. 

A review of the recent agendas of the evaluation teams highlights the 
nature of their work. The follow-up unit planned a long range survey on 
employment retention and other programmatic issues for the Summer Youth Program, 
the work experience programs, OJT, and pre-apprenticeship training. The 
manpower service center evaluation team planned a demographic analysis of the 
intake system, a geographic analysis of registration, service, and placement 
by census tract, and the development of a formula for calculating the rough 
dollar value of all services delivered to each census tract in the city. The 
training evaluation team planned to analyze "program hopping." As part of 
their normal responsibilities they also conducted exit interviews and tests 
for the LPN and clerical training programs. 
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Both the program monitoring and the evaluation work are used extensively 
by the senior staff for management decisions and for planning the mix of 
manpower services 9 the selection of service deliverers, and for contract 
negotiations. Unlike some prime sponsorships, the staff has not sought to 
make the results of their evaluations public, but they do use them in their 
discussions with the agencies, and the service deliverers undoubtedly know 
that the staff could always make them public if they wanted to. Thus, we 
found that most service deliverers were content to live with the findings from 
the evaluations; they did not question their integrity, but a few did have doubts 
about the accuracy and timeliness of the MIS on which the evaluation reports 
are based. 

Summary 

The three examples of prime sponsor monitoring and evaluation activities 
provide excellent evidence of the range of approaches used in CETA and 
illustrate some important themes. On one end of the continuum we have Bergen 
County's approach a small staff, limited quantitative data, a relatively 
weak MIS, use of volunteers on the MFC, emphasis on operational questions. 
On the other end, we have Baltimore Consortium a large evaluation and 
monitoring staff, extensive quantitative analysis of sophisticated MIS data, 
limited public involvement, emphasis on operational and long range questions. 
Yet, all three prime sponsorships reap significant benefits from their moni- 
toring and evaluation. All of them use the results in contract negotiations 
with service deliverers, to correct their operational problems, and to chart 
future directions for program expenditures. The particular approach 
chosen by a particular prime sponsor will depend on the overall size and 
complexity of the prime sponsorship, the political climate of subcontractors 
and interest groups, and the skills of the manpower staff. The experience 
of these prime sponsorships demonstrates the value of imaginative solutions 
to monitoring and evaluation. 

C. UNIVERSE OF NEED AND TARGET GROUPS 

In nearly all CETA programs in most communities there are far more 
people eligible for CETA than can be served by the program. This suggests that 
needy segments of the population must be identified, their size and characteristics 
estimated, and appropriate levels of service specified. CETA planners who 
are assigned the task of analyzing the "universe of need" face difficult problems 
that can be grouped into four somewhat overlapping categories: definitional, 
technical, evaluative, and practical. Each category poses a different kind of 
difficulty for prime sponsor staff. They will be briefly described below and 
approaches to coping with them, drawn from our research in several prime 
sponsorships, will then be presented. 

The first problem facing CETA planners doing a needs analysis is 
definitional: how should target groups be defined? The possible criteria are 
numerous: age, race, income, sex, geographic location, duration of unemploy- 
ment, work history, education, and so on. Target groups can be very broad 
categories, such as youth, heads of household, minorities, or narrow ones, 
such as "black female heads of household with two or more dependents." 

CETA legislation originally provided very minimal direction, and even 
In its latest revisions is only somewhat more specific about who to serve. 
The new emphasis is obviously on the long-term unemployed and the disadvantaged , 
but it is still not possible to serve all of them. In order to make sensible 



recommendations to senior staff, political officials, or planning council 
members who might participate in the selection of target groups , planners need 
to know the size and characteristics of the eligible population and where it 
is located in the prime sponsorship, at a minimum. Answering those questions 
can be very difficult, however. 

The second problem is evaluative. Beyond the base-line eligibility, prime 
sponsors should target their limited resources on those who can most benefit 
from the types of employment and training programs that they have available. 
This requires careful attention to the assumptions underlying one f s manpower 
programs. Target group selection, therefore, is a choice in manpower planning 
that, like so many others, cannot be made in isolation. 

Assuming that the prime sponsorship can reach agreement about how target 
groups should be defined, one must decide which groups will receive highest 
priority. In our view, the staff should place goals or quotas on particular 
groups and implement those decisions through the intake process and by designing 
programs that will meet the needs of the priority groups. If this is not done, 
then those enrolled may not be the ones that were "most in need," according to 
the prime sponsor's definition, nor the set of people who could best take 
advantage of the CETA programs in the prime sponsor's jurisdiction. 

Unfortunately, the debate over target group priorities is often conducted 
without reference to the evaluative issues and the data. Individuals and 
groups within and outside the prime sponsorship press claims for more service 
to their preferred group. Prime sponsor staff frequently are unable to respond 
to these claims effectively because they have little or no information on the 
size and needs of the groups being advanced. 

Determination about target group priorities should not be made solely by 
a competition of interest groups, though responsible political officials and 
staff members must ultimately make value judgments. Instead, such a debate should 
be informed by a careful analysis of the actual need of particular groups, 
broken down, not just by their ascriptive characteristics, but by their need for 
different kinds of manpower services. 

The third kind of problem is technical: how can prime sponsor staff make 
careful analysis unless they have adequate data on the universe of need? CETA 
planners are frequently frustrated in their attempts to obtain data that will 
allow them to analyze need for manpower programs in their prime sponsorship. 
Unfortunately, the data collection system of the U.S. Employment Service was 
not designed with the needs of CETA manpower planners in mind. Even though 
prime sponsor staff universely acknowledge that their data are imperfect, 
some have been able to reach more accurate descriptions of need than others. 
A point addressed in some of the examples below is how one can improve the 
quality of data within the limited resources of the prime sponsorship. 

The last type of problem is basically a practical one. CETA planners 
recognize that the goal of good target group planning is to help improve the 
quality of program operations and to assert the preferences of CETA decision- 
makers about who should benefit from CETA programs in a way that corresponds 
to a measure of local need. 

Elaborate needs analysis can be a waste of time and money if the results 
are not taken seriously by those operating the programs. Planners must take 
into account the limitations imposed by the types of CETA services that their 
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prime sponsorship is equipped and/or willing to provide, the history of past 
program performance with a particular client group, overall labor market 
conditions, and employer expectations. 

At the same time it is important not to let operational considerations 
dominate the planning process entirely. A balance between operational goals 
and goals for serving priority groups must be struck. 

Some Examples 

Denver, Colorado, The Denver CETA planning system does a particularly 
good job of handling the definitional and evaluative aspects of universe of 
need and target groups analysis. Their methodology was developed in 1976 
for the FY f 77 plan and used again for the FY '78 plan. It is based on a 
matrix that uses both population characteristics (ethnic group, age, sex, 
veteran status, and head of household status) and socio-economic indicators 
(percent of population, percent of labor force, unemployment rate, percent of 
group in poverty, and percent of applications at the Employment Service.) 
The economic indicators are assigned differing weights based on the judgment 
of the staff involved in planning (and ratified by the MFC). Indices of 
relative need are then derived for each population group. A second matrix is 
then created that cross-tabulates ethnic-sex groups with veteran, age and 
head of household status and subjective judgments are made about how to weight 
the frequencies. 

The index of need and the frequency index are combined to provide a 
single index. The twenty-five neediest groups were singled out as those 
about which decisions had to be made in terms of relative emphasis. The 
number of people in each of the twenty-five groups in poverty were calculated. 

The matrix method developed in Denver is not without its problems. It 
uses 1970 Census data, which are particularly obsolete in Denver because of 
its great growth in the 1970s. The staff plans to use more up-to-date data 
from the Employment Service on the characteristics of its registrants in 1977 
for the next plan, but some people will not accept ES data because they claim 
that the poor are reluctant to register there. Finally, many of the critical 
decisions about who gets served are still subjective. But, overall Denver 
has constructed a useful method for systematically examining its universe of 
need. 

Penobscot Consortium, Maine. Penobscot Consortium has done a particularly 
noteworthy job in supplementing the standard forms of quantitative data on 
universe of need with qualitative information, and in making good use of the 
quantitative data that they do have available. Wishing to avoid narrow target 
group designations, the staff has identified significant segments as the 
economically d is advantaged, youth, women, older workers, veterans, and native 
Americans. But they go beyond merely naming groups that should be served. 
The staff also conducts an exhaustive search of all available quantitative 
information in order to develop goals for service to each group in proportion 
to its representation in the universe of need. Other groups that do not loom 
large in such an analysis nevertheless receive high priority: migrant workers, 
offenders, handicapped persons, and alcoholics. 

The final goals for target group service are not based solely on the 
quantitative data. Qualitative assessment by the staff and advisory council 
members also plays an important role. The council considers the question of 
target group selection in some detail each year. For the most part, their 



input does not result in great changes in the staff research. However 9 they 
do offer suggestions that clarify and modify staff findings in productive 
ways. 

San Francisco, California. The San Francisco CETA program's approach 
to target group planning is impressive for two reasons. First, they vigorously 
attempt to serve people in their community in proportion to their presence in 
the unemployed population. Second, they are willing to design new programs to 
meet the needs of their clients rather than let the current mix of services 
dictate the kinds of people they can serve. The analysis of the unemployed 
population is relatively simple, the follow- through is unusual. 

The universe of need in San Francisco is very complex: there are many ethnic 
groupings, many language problems, and so on. Each year the staff analyzes 
the demographic characteristics of the unemployed population in the city. 
The findings are used in designing programs and in choosing service deliverers. 
Because San Francisco is so diverse, they have funded a large number of Title I 
programs over 30 in FY ! 78 to serve the many groups identified as being in 
need of manpower services. Each subcontractor is given specific standards 
about the characteristics of those that they must enroll in the program, and 
they are closely monitored by the staff. 

Unlike most prime sponsorships that work with a stable group of programs, 
services, and target groups from year to year, San Francisco's CETA staff has 
demonstrated a willingness to reach out for new organizations to deal with 
newly discovered or defined problems. There seems to be no significant 
barrier to employment that the staff is unable or unwilling to address. 
If the staff discovers that a program cannot deal with a specific problem, 
they will create a new program or establish linkages with other agencies that 
can assist in its solution. 



D. INTAKE AND ASSESSMENT STRATEGIES 

The previous section discussed methods for identifying, evaluating and 
establishing goals for service to target groups. In order to implement target 
group goals, it is necessary to establish a method for determining who gets 
into CETA programs and who doesn't (intake), and a method for channeling people 
into the "right" services or programs once they have been admitted to the 
CETA system (assessment). These two functions, which serve as the entry 
points into CETA, are clearly important determinants of what the system 
produces, whether the staff pays attention to them or not. This section 
addresses some of the central questions involved In developing effective intake 
and assessment strategies. 

There is an Inescapable tension built Into CETA programs that is resolved 
in one way or another at the point of intake and assessment. The baseline 
rationale for employment and training programs Is that' they provide services 
that enable certain people to obtain stable employment who would be unable to 
do so without some kind of interference in their ability to cope with the 
normal functioning of the labor market. Therefore, client need should be the 
primary determinant of whether someone receives CETA services. On the other 
hand, program administrators stress that CETA can only do so much to change or 
shape the employability of the people it serves, and these limitations must 
be recognized. This suggests that some particularly "needy" people may not 
be serviceable from the point of view of CETA. Thus, need alone cannot 
determine client selection. The problem is that It is easy to slip into the 
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practice of lf creaming ff once need becomes only ji factor in selection rather 
than the factor. 

The way in which most CETA programs are planned, implemented, and 
evaluated encourages creaming. Typically, planning periods are short, immediate 
evidence of results is demanded, and evaluations are based largely on positive 
termination rates and placement rates. The closer the clients are to job readiness, 
the easier it is to get a program off the ground, and to produce results that 
look good (at least if one doesn't look too hard). Thus, the incentives to 
cream the best participants from the applicant pool are considerable (program 
operators want capable people, DOL and elected officials want to see results, 
and so on) and the limitations of the programs are real. The intake system 
must somehow uphold the integrity of the CETA system by bringing in people 
who need help, and yet still contribute to the efficient operation of the 
programs. Our research has discovered many elaborate designs for achieving 
this goal, but few instances in which the designs were actually implemented. 

One means for making some headway in the search for an intake system 
that is both responsible and effective is to assert that program participants 
should reflect the priorities established in the analysis of the local universe 
of need. The desirability of linking client intake decisions to analyses of 
local need is generally recognized among CETA administrators, but again, seldom 
done well. Designing and implementing an effective intake system introduces 
some difficult problems that go far beyond this simple recognition. Many 
prime sponsors base service levels either wholly or partly on past levels of 
service to specific target groups. This practice obviously introduces a 
great deal of circularity in the relationship between target group planning 
and participant service, which is tidy, but hardly justifiable. In some cases 
this practice is followed to reduce uncertainty, especially with regard to 
meeting standards set by national and regional offices of DOL. In other cases 
this practice comes about because service levels for different target groups 
have emerged after years of political compromises among local clientele 
groups that cannot be changed easily. 

A second problem that must be dealt with is whether to impose strict 
service quotas on those responsible for intake in order to achieve the 
desired mix of clients, or to leave some flexibility in the intake system. 
In the course of our research we have found that prime sponsor intake systems 
show a wide range of variation on this score, from those who admit applicants 
randomly (usually after eligibility determinations) , to those in which heavy 
controls are imposed on the intake system. 

It is apparent that the two problems discussed above are related. If 
next year's service goals are determined by last year's levels of service there 
is clearly no need to impose intake quotas. But, assuming that the specified 
goals for target group service call for at least some departure from the past, 
it may be necessary to introduce some mechanisms for stimulating intake people to 
depart from existing operating procedures. A quota system may be one way of 
doing this. 

The examples discussed below will focus on intake strategies that function 
as part of a concerted effort to implement concrete client service goals where 
that goal demands that those responsible for intake employ some type of standard 
in a systematic way to select participants from the applicant pool. This means 
that our examples will be drawn from prime sponsorships in which some 
positive and definable objective is assigned to intake, and where that objective 
appears to have been successfully carried out. 
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Effective Intake Strategies: Some Examples 

The King-Snohomish Consortium, which serves the Seattle SMSA, employs an 
intake system that is tightly controlled and effective. The intake, assessment, 
and referral functions are centralized in the sense that the Washington state 
ES is responsible for providing these services to all CETA applicants in the 
consortium, but is also decentralized in the sense that ES operates 11 offices 
for this purpose. The KSMC staff, the sub-contractors, and ES have worked 
very closely together for a number of years to develop and refine this intake 
system, which was initially based on an ES design. 

The primary objective of the KSMC intake system is to make sure that 
target group service goals are met during the program year. The planning staff 
of the KSMC, in conjunction with the advisory council, establishes the 
target group goals. In recent years broad target groups have been preferred 
(minorities, females, and youth for FY 1978). Prime sponsor-wide goals are 
broken down by individual sub-contractor, and service goals are set for each of 
them. The ES offices, aware of the service goals of each subcontractor, make a 
concerted effort to refer clients that will allow subcontractors to meet their 
goals. This procedure also guarantees that overall service goals are met. 
The sub-contractors have the right to reject ES referrals, but this right is 
rarely exercised. 

Meeting the target group goals of the sub-contractors is not, of course, 
the only factor considered by ES in making referrals. They try to refer 
people who have been waiting for the longest period of time, and those whose 
assessment results suggest that they would benefit from the program offered by 
the sub-contractor. 

The San Francisco prime sponsorship f s client selection system 
combines flexibility with a firm commitment to achieve certain target group 
service goals. The staff and the advisory council identify a number of 
significant segments (in recent years 12 different significant segments have 
been isolated) . Some of these target groups have guaranteed levels of 
service assigned to them. This guarantee means that the minimum level of 
service assigned to these groups will be met during the year, even if it 
means interrupting the normal flow of client selection. In FY 1978, the 
economically disadvantaged Latinos, Chinese, welfare recipients, Filipinos, 
American Indians, ex-offenders, older people and those with limited english 
speaking ability were all listed as "minimum-level" target groups. For the 
other significant segments (Blacks, youth, and Vietnam Veterans for FY 1979) 
no fixed levels of service are guaranteed. 

The San Francisco CETA system is decentralized, with most sub-contractors 
taking care of their own recruitment and selection. Those selected by the 
sub -contractors are then sent to the State Employment Service (JSO) for 
certification and formal enrollment. However, when monitoring reports show 
that a minimum level group is in danger of not meeting its service goal, the 
JSO, under the direction of the prime sponsorship staff, can correct any 
imbalances by imposing tight controls on client selection. This occurs very 
infrequently, but the mechanism is present when needed. 

Another way to promote the achievement of client service goals through 
the intake process is to establish a rating system for applicants that rewards 
applicants who have certain characteristics. The Cleveland Area Western 
Reserve Manpower Consortium has established such a system for its PSE programs. 
Points are given to applicants who fall into certain high priority groups such 



as racial minorities, women*, handicapped, Hispanics, those with less than a 
high school education , those with limited English speaking ability, and those 
who have been unemployed more than 40 weeks. Points are also given to those 
who fall into significant segment categories youth, economically d i sad van t aged, 
welfare recipients , and persons over 40 years of age. The order of referral 
to PSE positions is then governed by the scores applicants receive when these 
points are totalled up, on their intake form. This helps to give high priority 
groups an edge in the competition for PSE jobs. 

The Baltimore Metropolitan Manpower Consortium uses its performance 
contracts to reduce some of the incentives to "cream" at the intake level* 
Sub-contractors are offered monetary bonuses for successfully training and 
placing participants with particularly low prospects of finding employment 
on their own. These "hard-to-train" participants are identified by very 
weak educational backgrounds, and low scores on aptitude tests. Thus, many 
sub-contractors encourage referrals of this type, which is precisely the 
opposite of what occurs in most prime sponsorships. 

Another method for achieving participant service goals that lessens the 
tendency to cream is establishing eligibility standards that exceed those 
contained in the regulations. Under the new CETA legislation, eligibility 
has been more narrowly defined than ever before. The prime sponsors discussed 
below had already successfully taken steps to restrict eligibility before 
passage of the new CETA. 

In Bergen County only low income applicants are accepted in both Title I 
and PSE programs. Atlanta requires that all Title I participants enrolled in a 
component that pays a wage or a stipend be economically disadvantaged . The 
Penobscot Consortium has a general rule of only accepting economically dis- 
advantaged persons in both Title I and PSE programs, but exceptions are allowed 
under "special circumstances." San Francisco has established its own measure 
of economic disadvantage. Instead of using the 70% of the BLS lower living 
standard, they use the OMGB-approved Orshansky poverty level. The difference 
between the two measures is significant. For a family of four the Orshansky 
poverty level is an annual income of $5850, while the 70% of the BLS lower 
living standard figure comes to $7640 in the San Francisco area. Syracuse 
requires that Title I participants be unemployed for at least 30 days prior to 
entering the system, rather than the 7 day period mandated in the regulations. 
These examples demonstrate that it is possible to implement CETA programs 
effectively with relatively narrow eligibility standards. Although the new 
national standards are more restrictive than before, they are still inadequate 
definitions of need in most prime sponsorships. Therefore, most prime sponsors 
will wish to add restrictions that more carefully target their programs to 
people who they define as most in need. 

The city of Syracuse has also been able to relieve some of intake 
pressures by setting up a two-step intake process. Three agencies in the 
city are involved in screening CETA applicants, one for the elderly, another 
for minorities, and the ES for everyone else (including minority and elderly 
people who do not go to the other agencies) . The bulk of the applicants end 
up going through ES, which has worked very closely over a number of years 
with the Syracuse Employment and Training Agency (SETA) an in-house assessment, 
counseling, referral, training, job development and job placement operation. 
ES intake people, therefore, are well versed on SETA assessment criteria, which 
become the final determinant of what services, if any, an applicant receives, and 
they use these criteria to guide them in making intake decisions. This greatly 
reduces the number of people who appear at the SETA office requesting services, 
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so the SETA staff has an excellent opportunity to engage in extensive 
assessment and counseling with those who do come to them* (The Syracuse 
assessment system is described in more detail in the next section.) This 
two step intake process helps to reduce the pressures to cream on both the 
screening agencies and the SETA office. 

Assessment 

Creaming also occurs within CETA systems with the "best 11 clients 
routinely sent to specific programs. Some prime sponsors have certain 
programs (usually work experience) that may be used as a "dumping ground" 
for people who are accepted into the system, but are judged to be unable to 
perform effectively in skill training, OJT, or PSE programs. In some 
cases this is appropriate since a sequence of programmatic services is 
planned for the client, but in other cases no such plans exist. This, again, 
points to the presence of a built-in tension between the demands of program 
operators to have people they feel they can work with effectively, and the 
needs of clients with the least hope of finding a job. This tension is 
resolved for better or for worse through client assessment. 

There is a fair amount of consensus among CETA administrators that 
client referral decisions should be informed by both testing and 
counseling, and that the end product should be some sort of employability 
development plan a prescription of the services that a client should receive to 
become job ready. The nature of these plans varies, as does the extent to 
which they are used by those making referrals. The section that follows 
presents a few innovative and effective approaches to client assessment. 

Innovative Approaches to Client Assessment: Some Examples 

The Bergen County Community Action Program (BCCAP) has developed a 
multi-faceted and effective client assessment component for their CETA system. 
During the intake process an initial determination is made whether an applicant 
is in need of CETA services. If this is judged to be the case the person is 
referred to an employment and training counselor, who works with the client 
to develop an employability development plan (EDP) * Several kinds of assessment 
may go into the development of such a plan. One is simply in-depth counseling, 
or "person to person" assessment. Another is educational testing, such as 
GATE or other standardized tests, that provide information on a client's basic 
reading, math, and verbal skills. Finally, and most interestingly, many 
clients are referred to a Career Decision Lab, where they are paid an allowance 
while being exposed to various career possibilities. The lab is designed- to test 
each person's interest in, and aptitude for, different employment careers. 
This assessment tool is especially useful for clients who have no firm 
career preferences when they enter the CETA system. The final EDP, which 
outlines the services a client needs to become job ready is a product of the 
results of the different assessment procedures, the judgments of the counselors, 
and the preferences of the client. 

The key to client assessment in the eyes of the Syracuse Employment and 
Training Agency is the personal touch. They begin by assuring applicants that 
employment references and tests will not be used to exclude them from the 
program, but only to pinpoint areas in need of development. They employ a 
limited number of tests. GATE is used for everyone, and other tests are used 
to inventory some clients 1 interests, and manual dexterity. 



88 



The heart of the assessment process in Syracuse is a 4 to 8 week one-on-one 
counseling activity for each client. They try to match client and counselor 
very carefully so that these sessions can be most fruitful. These 
counseling sessions are open and frank. Seventy-five percent of the applicants 
complete the counseling and are enrolled in the program. Participants are also 
exposed to interview boards composed of representatives of private sector 
employers, which helps to prepare participants for real interview situations, 
and better to assess the areas where improvement is needed. All of these 
different components make up an extensive client assessment function, which is 
viewed by SETA staffers as one of the strongest points of their program. 
The end product of the assessment process is a prescribed sequence of services 
for each client, and a signed contract that commits the prime sponsor to 
providing, these services. 

One of the most systematic attempts to structure client assessment in a 
manner that identifies gaps between a client's interests and skills and the 
requirements of the labor market has been established by the Penobscot Consor- 
tium. One striking feature of their client assessment procedure is the 
conscious emphasis on bringing the demands and expectations of future 
employers into the assessment stage of CETA. In this way the "human capital 
gap" fot each client can be clearly delineated. Most of the specific 
components of what the PCTEA calls a "participant assessment matrix," which 
they develop for every client, have been mentioned in the discussions above. 
The structural characteristics (age, sex, race), educational and training 
experiences, employment history, interest profile, basic and job related 
skills, and behavioral characteristics (appearance, motivation, punctuality) 
of the clients are assessed and listed on one form, so that their personal 
barriers to employment can be easily identified. Once the matrix is completed 
an employability plan that prescribes the service appropriate for remedying 
the identified employability problems is developed and implemented. 

E, LABOR MARKET ANALYSIS 

One of the often cited advantages of decentralized (as opposed to 
categorical) employment and training programs is that they greatly enhance 
the ability of planners to design programs that correspond to local labor 
market conditions. This, of course, suggests that prime sponsors can and do 
conduct meaningful labor market analyses, and that this research is used in 
making decisions about program design and operations. However, we have 
observed that much of the large quantity of labor market data that has been 
collected or generated by CETA planners is used only for the purpose of 
satisfying DOL grant application requirements. There is often very little 
integration of labor market information and program planning. 

The fact that labor market analyses are frequently put to very little 
use by those designing and operating CETA programs may result from poor staff 
organization, the insensitivity of labor market analysts to the needs of 
operations staff, or the unwillingness of the senior staff to be guided by 
the results of labor market analysis. 

In many cases however, labor market analysis is ignored because of data 

limitations. Although labor market data are usually available in large quantities 3 

the data are often of such poor quality that it cannot yield answers to 

questions of primary concern to CETA planners. Most prime sponsors rely on ES 

labor market trend data, newspapers, national economic indicators, and labor 
market studies completed by other local organizations (such as Chambers of 

Commerce and regional planning commission) to project labor market trends. 



It is common for the program planners, the operations staff, and the 

labor market analysts themselves to agree that limitations in these data make 

them a very unreliable base for making programmatic decisions. 

This recognition leads many prime sponsorships to look elsewhere for 
more reliable labor market information* Judgments offered by different local 
"experts" (or other contacts) are sought and often used instead of the 
quantitative projections to inform programmatic decisions and placement 
strategies. In some cases careful efforts are made to check the reliability 
of these qualitative data sources, and make those with proven records of 
accuracy part of the formal labor market information system* In other cases, 
judgments of varying degrees of reliability are received in a seemingly random 
fashion, and exert a very unsystematic influence on decision-makers. 

In this section innovative approaches to the collection and use of 
quantitative and qualitative labor market information will be described. 
A necessary condition for any approach to be designated as innovative is that 
it be used to help determine the design and operation of CETA programs. Many 
very elegant analytical routines exist that have not been shown to be useful 
in practice, and therefore will not be included in this report. In addition 
to proven usefulness we are interested in approaches that are both unusual 
and imaginative. 

Innovative Quantitative Approaches to Labor Market Analysis: Some Examples 

Probably the most systematic attempt to use quantitative labor market 
information to inform decisions about training priorities that we have 
observed is a system put in place in the spring of 1978 by the Denver 
Manpower Administration (DMA) . Their approach allows the DMA planning staff 
to go beyond the simple identification of areas of employment demand, and 
pinpoint specific areas of employment growth most suitable for CETA trainees. 
This is done by employing a number of different data sources. The Annual 
Planning Report issued by the Colorado Department of Labor and Training is 
used to identify growth occupations for the upcoming years (growth is defined 
as 100 or more openings, or a thirty percent growth rate over the next five 
years). Last year 138 growth occupations were identified. These growth occupa- 
tions are then checked and double-checked by consulting other labor market 
data the DMA staff has available. These alternative data sources include 
employment projections and lists of unfilled openings developed by the Department 
of Labor and Training, reports on worker demand and supply from the Board for 
Community Colleges and Occupational Education, and want ads in local newspapers. 

In conjunction with these data that identify demand occupations, detailed 
information on average hourly wages and advancement opportunities in each 
designated occupation is assembled by the DMA staff from a wage survey and the 
Occupational Outlook Handbook published by the Bureau of Labor Statistics. 
This information is used to rank demand occupations according to the benefits 
they appear to offer to CETA trainees. The ranked occupations are then related 
to the requirements (education, experience, training, etc.) they place on 
applicants, and the rate of personnel turnover is considered before the staff 
comes up with a final index of high priority areas for CETA training. The DMA 
hopes that this system will help them correct some deficiencies in the training 
component of their Title I program. 

A number of other prime sponsors employ quantitative data sources to 
project job opportunities that go beyond the "standard" sources of labor 
market information mentioned before. Some of these data sources are available 



only in particular prime sponsorships , but others could presumably be of 
benefit to many other prime sponsors. A few of these will be discussed below. 

The Omaha Consortium commissioned the Center for Applied Urban Research 
at the University of Nebraska at Omaha to do a series of labor market studies 
covering the Omaha SMSA. In one study local employers were surveyed to 
determine growth occupations in the SMSA, and more specifically to identify 
openings in entry level and paraprofessional positions and sources of 
training for these jobs* The Center also did a more general study of the 
structure of the local labor market by industry and occupation for the 
consortium. This study included projections of areas of growth, and covers 
the period from 1970 to 1985. Many other prime sponsors have contracted with 
local colleges or universities for studies like these, and it seems to us to 
be an option that has wide potential applicability. 

The Penobscot (Maine) Consortium has taken a couple of interesting steps 
to improve the quality of the quantitative labor market data with which they 
have to work. They have used Occupational Employment Surveys conducted by 
the Maine Department of Employment Security in 1973 and 1977 to trace labor market 
trends in a number of Maine industries over time. The consortium also 
strongly supported a proposal before the Maine SMSC to fund a statewide 
survey to gather information on youth employment patterns. When the study 
was completed the consortium convinced the University of Maine the contractor 
for the survey to break out the results for the Penobscot Consortium. They 
were thus able to obtain up-to-date labor market information at a relatively low 
cost. 

In Syracuse the Community Council on Careers in Greater Syracuse makes 
available a series of reports that project job openings in manufacturing firms 
based on a survey of local employers. The prime sponsor staff uses these reports 
to supplement the more standard data sources they have available in the 
compilation of a yearly Labor Market Report. 

These examples are intended to illustrate the point that prime sponsors 
can usually improve the quality of quantitative labor market data they use 
in planning either by making a systematic effort to seek out and tap existing 
data sources in their localities (as in Syracuse and Penobscot) , or to take 
steps to see that better data are generated (as in Omaha and Penobscot) . 
These steps do not have to be expensive or time consuming if the options are 
thought through clearly (some prime sponsors have funded PSE projects to 
survey employers, which can be an inexpensive way to collect labor market 
data). The improvement in labor market information can be substantial. 

Innovative Qualitative Approaches to Labor Market Analysis; Some Examples 

The Baltimore Metropolitan Manpower Consortium has established an 
impressive system for using qualitative information on job openings and labor 
demand projections obtained from members of local labor unions and business 
firms in labor market planning. This information ultimately becomes a major 
determinant of the types of skill training programs the consortium will offer in 
any given year. 

The prime sponsor staff has worked with members of the local business 
community and labor unions as well as representatives of local schools to 
create 15 Labor Market Advisory Councils. The Councils are organized around 
distinct employment and industry categories (for example, building trades, 
mechanics, ship building, cooking) and they perform a number of functions, 



which help to improve the quality of the labor market information the staff 
has available. Every year, during the planning process the LMACs meet as a 
group and react to staff -developed labor market projections that already 
incorporate various data sources including a survey of local employers, 
Employers on the councils describe their hiring plans in the areas identified 
and provide further details such as the kinds of qualifications they will 
require, the wages that will be offered, and the advancement possibilities 
associated with different positions. Union and other labor representatives 
report on the kind of training the jobs require, working conditions, turnover 
rates, and related matters. The basic mission of the LMACs is to supplement 
the quantitative projections developed by the staff and pinpoint those occupations 
that the CETA system should address. 

In addition to this very specific and formal function the LMACs perform 
during the planning process, members of the councils help the staff at various 
times during the year by reacting to late-breaking plans, reviewing proposals 
for training programs, and evaluating on-going training programs. Thus, the 
councils act as both a formal deliberative body (meeting at least two or 
three times a year) and an informal network of contacts whose knowledge and 
services are frequently requested when the staff faces difficult problems 
for which their expertise is particularly appropriate. 

In Syracuse a similar, but less elaborate process for integrating 
public input and staff projections in an analysis of the local labor market 
has been established. When the staff completes its yearly Labor Market 
Report it is referred to the Research Committee of the MFC. Committee members 
offer the same sorts of observations that the members of the Baltimore LMACs 
provide, and this information is used in decisions relating to the kinds of 
training programs the city will offer during the next year. 

The Albuquerque consortium goes even farther than Syracuse in the effort 
to incorporate the services of advisory council members in labor market analysis. 
The Economic Development and Labor Market Task Force of the Consortium Planning 
Board is responsible for working with labor market data to determine areas of 
employment opportunities for CETA trainees. They are, of course, assisted in 
this effort by the staff, but the staff functions mainly in a technical 
assistance capacity, and does not offer a great deal of substantive input. 
The task force members themselves are, therefore, the major actors in the labor 
market planning process. 

The most common form of qualitative information on labor market conditions 
used by CETA planning staffs is supplied by CETA job developers. While most 
prime sponsors incorporate the observations of job developers into their labor 
market analysis some do so more systematically and effectively than others. 
The Bergen County Community Action Program, which administers all the CETA 
programs in the county, has evolved a very effective process for feeding the 
observations of job developers back into the planning process. A centralized 
job development unit carefully divides the county into sections and contacts 
all relevant employers four times each year. The primary purpose of this 
activity is to locate immediate jobs for CETA participants, but a very 
important secondary purpose of this exercise is to develop information on 
current and future demand occupations. Job developers also inform planners 
about the areas of skill training in which they have encountered difficulty in 
placing people, and those in which placements have been easily secured. All 
of this information is used in an effort to make the best possible choices 
about the types of skill training programs that should be offered. 



F SYSTEM DESIGNS SERVICE DELIVERER SELECTION, SERVICE DELIVERY SYSTEMS , 
AND PERFORMANCE CONTRACTING 

One of the major decisions prime sponsorships make is the selection 
of service delivery agencies. There is always potential for a fair amount 
of conflict to arise in the course of making these choices; no prime sponsor 
welcomes the kind of petty bickering and infighting among rival service 
deliverers that can and often does accompany funding decisions. Consequently, 
most prime sponsorships develop some method for reducing this type of conflict. 
However , some prime sponsorships also recognize that a little competition 
between service delivery agencies may lead to the -.improvement of the system as 
a whole, and make some effort to promote what they hope will be healthy 
competition* 

The importance of service deliverer funding decisions depends on the 
extent to which the service delivery system is open or closed. Many prime 
sponsorships effectively eliminate conflict over service deliverer funding 
decisions by maintaining the same set of service delivery agencies at more or 
less the same level of funding year after year. In such cases funding 
decisions in any given year tend to be routine, incremental, and not particularly 
interesting or important. In other cases the system is open, at least to some 
degree, which means that funding decisions are quite significant, and the study 
of how some prime sponsors make these choices can be useful for prime sponsorship 
staffs. 

"While the way in which service deliverer funding decisions are made is 
often important, the results of such decisions are even more important. Every 
aspect of a CETA system is affected by the number and type of agencies that 
are selected to operate employment and training programs. Not surprisingly, 
variation in the extent to which CETA programs are subcontracted to outside 
agencies, and the kinds of agencies receiving the subcontracts is great. 
Some prime sponsorships are convinced that operating all or most of the programs 
within their own organizational structure is the only way to make the system 
manageable and responsive to changing policy standards and economic conditions. 
Others believe that a mix of in-house and subcontracted programs is desirable. 
Still others are firmly committed to the idea that the prime sponsor staff 
should be purely administrative, and the responsibility for operating programs 
should be sub-contracted* 

In this section innovative ways of chosing among competing service deliverers 
will be described. We include approaches that have been observed to work in 
situations in which the decisions are recognized by everyone as consequential 
ones. By "consequential" we mean that the possibility existed for agencies not 
receiving funding in the previous year to be funded for the following year, or that 
current service deliverers might be defunded, or that changes in the operational 
responsibilities, and funding levels for an existing set of service deliverers 
might be made. 

Exemplary models of service delivery systems will also be described. The 
models presented will represent the three general models referred to above: 
those in which all or nearly all the programs are operated in-house, those in which 
a mixed pattern of in-house and sub-contracted program operation exists, and 
those in which all or nearly all the programs are subcontracted to outside 
agencies . 
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Exemplary Service Deliverer Selection Processes 

Many prime sponsors have decided that the best way to make rational 5 
open, and performance-oriented service deliverer funding decision, while 
keeping conflict among service deliverers under control 9 is to institute a 
formal Request for Proposal (RFP) process* In most cases , the MFC, or 
some public body outside the staff, is brought in to review the proposals and 
make funding recommendations so as to reduce the chance that favoritism 
(or the perception of it) will creep into the selection process. Our view 
is nbt that a formal RFP is a panacea for the problems inherent in service 
deliverer funding decisions , but we have observed a few that seem to work pretty 
well. 

Many problems have beset those who have attempted to use an RFP process to 
make service deliverer funding decisions. In some cases, the process is 
viewed by all parties involved as an unnecessary burden because it has no 
impact on the outcome of funding decisions* In other cases political influences 
interfere in a way that distorts the selection process. Still other prime 
sponsors have found that the introduction of an RFP greatly expands the 
scope of conflict over funding decisions, since new groups that previously 
had no knowledge of, or interest in, CETA are activated by the public RFP. 

The Atlanta prime sponsorship established (after a number of years of 
experience) a carefully constructed and systematic service deliverer 
selection process. The staff initiates the process by drafting a Request 
for Quotation (RFQ) , which invites agencies in the city to submit proposals 
for specified services. The RFQ replaced a more general RFP when it became 
a burden because it was too open ended. The RFQ not only provides a general 
outline of the kinds of services requested, but also supplies detailed 
program requirements. The draft RFQ is reviewed by all units of the prime 
sponsor staff, and the final version is mailed to all agencies on their 
bidder's list and the city council. It is also advertised in local media. 
Potential contractors are given two weeks to submit proposals; the proposals 
are reviewed the following week. 

Reviewing proposals and making funding decisions is the responsibility 
of subcommittees of the advisory council. These subcommittees are organized 
around functional categories (intake and assessment, skills training, OJT, and 
so on), and appointments are made so as to avoid any conflict of interest. 
Funding recommendations are based on a rating system used by the subcommittees. 
The system takes into account various factors, including past performance. 
Subcommittee recommendations are passed on to the full advisory council, 
which has to reconcile the different subcommittee recommendations with tht 
reality of the expected budget. The job has recently been made easier by 
designating some services as "core" components and others as "peripheral." 

As the service deliverer funding process unfolds each year in Atlanta 
the staff is very much involved by providing data and recommendations to the 
subcommittees and the full council, but appears to let the advisory groups 
make the final recommendations on their own. The Mayor, of course, has the final 
authority to choose service deliverers, but he has rarely used this authority 
to overturn advisory council recommendations* It seems clear that the 
existence of this elaborate formal process makes the Mayor more reluctant 
to act against the wishes of the council than would otherwise be the case. 

The San Francisco prime sponsorship employs a similar process, but 
the staff of the Mayor's Office of Employment and Training (not the MPC) 
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eviews the proposals received in response to the KFP. A task force composed 
f various tsnits of the MOET (fiscal, monitoring, planning, contracts) is 
reated each year to conduct the reviews. Members of the task force do not 
ommunicate with agencies submitting proposals during the entire period in 
hich the proposals are being reviewed so as to avoid any charges of favoritism, 
his review results in a ranking of the proposals based on a standardized list 
f criteria including, where applicable, past performance. The original propo- 
als, plus the staff rankings, are then sent to the Evaluation Committee of 
he Advisory Council, which reviews the material, and conducts public hearings 
or proponents of the programs who wish to plead their cases. Thus, the 
ouncil performs the buffer function, but the staff is able to interject 
ore of their acquired expertise into the review of program proposals. 

The public RFP route is not the only way to structure a good and workable 
ervice deliverer selection process. The Baltimore Metropolitan Manpower 
onsortium has also used its annual selection process to introduce new programs 
nto the system, alter the operational responsibilities of some operators, 
nd to press all the agencies with which it subcontracts to achieve better 
erformance. The selection process they employ, however, is in many ways the 
pposite of the public RFP. The MOMR staff has long insisted that service 
eliverer selection and funding decisions are best viewed as technical 
ecisions, and should, therefore, be primarily a staff responsibility. Thus, 
n Baltimore, agency funding decisions are not issued for public debate 
ad political squabbling, but are handled in private negotiations between the 
taff and the prospective service delivery agencies. 

This type of approach can work in Baltimore because it is complimented 
y many other components of their CETA system, many of which have been 
Iscussed in other sections of this report. The MOMR has excellent labor 
arket information and uses performance contracts, which are backed up by 
strong monitoring and evaluation effort. Contractor performance is 
tioroughly assessed each year. When it comes time to make service deliverer 
unding decisions the staff has data on both the need for a particular service 
nd, where applicable, the past performance of the subcontractor proposing to 
ffer it. 

gemplary Service Delivery Systems 

In-House Systems of Operating Responsibility. Probably the strongest 
sample of an in-house delivery system we have observed is the Penobscot 
onsortium in Maine. Many of the oft-praised (but seldom observed) aspects 
f the "ideal" CETA system are present in the PCTEA. The program is compre- 
snsive, integrated (both administratively and programmatically) , effective, 
ell-managed, and responsive to local conditions. This situation has come 
bout because a dedicated, hard-working and imaginative staff has been given a 
reat deal of flexibility (indeed almost experimental conditions) to design 
id implement a model employment and training system. 

The core of the delivery system is the four Offices of Training and 
nployment Programs (OTEPs), which are strategically located throughout the 
rea served by the consortium so as to provide access to employment and training 
ervices to everyone in the consortium area. Each office is in a position to 
Efer clients the full range of CETA services (assessment, supportive services, 
BE, WE, CT, OJT, PSE, and so on). Employability developers work with 
Lients to establish employability plans and continue to serve them in a 
Dunseling capacity as they proceed through the prescribed sequence of services. 
t is common for the employability plan to call for the client to participate 
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in more than one of the traditional programs (WE, OJT f CT, PSE) . In most 
cases a client f s exit program (which could be OJT, CT, or PSE) includes a 
commitment to hire by an employer, if performance is satisfactory 

Despite recent movement at the national level in the direction of 
re-establishing a more categorical approach to employment and training 
programs, the Penobscot consortium has maintained a tmified delivery system. 
Intake, assessment, counseling, referral, and placement for almost all the 
programs take place in the OTEPs. The only exception is project PSE, which 
is taken care of through the local ES office (this and a small OJT 
subcontract with the local AFL-CIO are the only non-in-house aspects of the 
system). Thus, Penobscot has been able to implement what many other prime 
sponsors are only able to conceptualize in abstract terms. The different 
components of CETA are not only viewed and planned as if they formed a unified 
whole, they are actually operated according to this vision* 

The activities of the OTEP offices are centrally coordinated by the Area 
Operations Department of the PCTEA. In addition to the intake, assessment, 
counseling, and referral responsibilities just described, the Area Operations 
Department oversees and directs staff members working on the development and 
coordination of OJT and PSE positions, skill training classes, and work 
experience sites. An especially noteworthy feature of the Penobscot system 
is that they have not sunk costs into ongoing training programs so as to 
maintain the flexibility to shop around to find opportunities for class size 
training that fit local labor market conditions. A smaller, but independent 
staff unit, the Office of Private Sector Initiatives, provides valuable assistance 
to the Area Operations Department in such activities as client assessment, 
training workshops, and the development of CT and OJT positions. The Area 
Operations Department also maintains close working relationships with all the 
other staff units within the PCTEA. These include the Planning and Fiscal 
Departments and the Office of Policy Evaluation and Research. 

Mixed Systems of Operating Responsibility. The Syracuse prime sponsor- 
ship successfully operates its CETA programs using a mixed system of in-house 
and sub-contracted operating responsibility. Early in CETA the staff 
settled on the idea of operating the key components of the system in-house, 
while sub-contracting some ancillary services to outside agencies. The 
one exception to this pattern is a continuing subcontract with the Board of 
Education to run the Skill Center, which is clearly a key program component. 
The staff believes that by maintaining direct control over the operation of 
most key aspects of the system it can insure basic standards of quality, 
provide a clear sense of direction, and encourage integration with other programs. 
Sub-contracting some parts of the program was considered important because 
the staff wanted to encourage some agencies, which were providing services 
outside the CETA system to participate within the system, to give important 
client groups an agency perceived to be sensitive to their needs, and simply 
to take advantage of some agencies whose competence had been proven through 
past performance. 

Assessment, OJT, WE, individual referral, and job development and placement 
are run in-house. A former Model Cities agency does intake and recruitment in the 
minority community; another community organization does intake, recruitment, 
and job development for older workers. The Employment Service performs a general 
recruitment and intake function. Remedial education is the responsibility of the 
Education Opportunities Center connected with the state university. Finally, 
the School Board runs the skill center. Negotiations between the skill center 
and the staff have been serious and prolonged. The nature of the Center's 
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operation has changed substantially since the beginning of CETA, mainly as 
a result of the constant prodding of the OFSAC staff. In this case the basic 
determination of the staff to have the key components of the system operated 
according to their standards has extended beyond the programs run in-house to 
include the sub-contracting agencies. 

While Syracuse is an example of a mixed system of operating responsibility 
that is weighted slightly in the direction of an in-house model, the Baltimore 
system is a mixed system that favors the sub-contractor model: many of the 
core programmatic functions are sub-contracted to outside agencies. All intake, 
assessment, and referral, and part of the job development and placement 
responsibility is located in the 21 manpower service centers (MSCs). Most of 
these centers are run by outside agencies (community based organizations, ES, 
and county governments) , but a few are operated directly by the MOMR. Skill 
training is sub-contracted to several training organizations in the consortium 
area. These training sub-contractors also have job placement responsibility 
for their trainees. The MGMR operates OJT, PSE, most WE, and a central job 
development service in-house. 

The reasoning behind this set-up is fairly straight-forward. The decision 
to set up the network of MSCs was based on the desire to take advantage of 
the strong neighborhood ties of many community based organizations in the city, 
and to utilize existing services (ES and county government) in the counties. 
In choosing training sub-contractors the MGMR favors organizations with direct 
employer contacts, because these types of training outfits have in the past shown 
themselves to be most effective in providing quality training and securing 
placements. The in-house operation of OJT and WE came as a result of the 
inability of previous sub-contractors in these areas to perform up to staff 
expectations. Similarly, the central job development unit was established in 
response to perceived shortcomings in previous job development efforts. The 
success of this system of operating responsibility is heavily dependent on the 
use of performance contracts, and the extensive monitoring and evaluations carried 
out by the MOMR staff. 

Systems of Sub-Contracted Operating Responsibility. The San Francisco 
prime sponsorship subcontracts all Title I operating responsibility. During 
FY 1978, more than 30 agencies provided CETA services under Title I in the city, 
with many of the agencies delivering more than one service. Eighteen agencies 
operated classroom training programs (including english-as-a-second-language) , 
6 provided OJT, 4 ran WE, and 8 worked on other activities. The San 
Francisco staff believes that the sub-contracting model is the best way to 
operate a CETA system in San Francisco for two basic reasons. First, utilizing 
a large number of service delivery agencies allows the diverse and numerous 
constituency groups that exist in the city to be represented in a very concrete wa] 
within the system. Twenty of the 30 sub-contractors are community based 
organizations that represent local constituencies. Second, the San Francisco 
staff recognizes that with a large number of experienced deliverers with proven 
records of performance, it would be senseless to exclude them from the system. 

The central component of the San Francisco system is a central Manpower 
Services center operated by the Job Services Office (JSO) (ES in California). 
This office certifies all applicants, does intake and recruitment for some 
service deliverers, handles all supportive services, job development and place- 
ment. The JSO also verifies all terminations and placements. The other programs 
and services are provided by many deliverers. The largest single program is 
classroom training, which takes place at the skill center. 
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The reason that this wide-ranging operation can function efficiently and 
effectively is that the MOST staff maintains firm administrative control over 
several critical aspects of the system. For example 9 service deliverers are 
not assured of funding for the future. For existing deliverers past performance 
is the primary criterion taken into account when service deliverer funding 
decisions are made. Since the beginning of CETA several agencies have been 
dropped, or had their operating responsibilities changed 9 and a number of 
new agencies have been brought into the system. Probably the most important 
means of control are the performance contracts and an elaborate monitoring 
system. 

Performance Contracting 

Most prime sponsors use either a mixture of subcontracted and in-house 
manpower services or subcontract everything. There are a number of sound 
reasons for subcontracting, including the use of programs of demonstrated 
effectiveness, the ability to change program mix through altering the types 
of subcontractors chosen, and the advantages of using organizations with 
established clientele groups. 

However, having a lot of subcontracted services can also create problems. 
Coordination of manpower programs may be difficult to achieve. There may be 
a long lag time between the introduction of changes by central staff and response 
by subcontractors. Program operators may cream the applicant pool. One 
particularly effective method for countering these problems is an approach 
known as performance contracting. 

In their simplest form, performance contracts provide for reimbursement 
of program operators when specific services are completed or when particular 
goals are achieved. They may be highly detailed and tailored to the work of 
a particular contractor, or broad and applicable to all service deliverers. 
The central purpose of the contract is to influence the behavior of the 
subcontractor in desirable ways determined through negotiation with the 
contractor. While several prime sponsors use one or another form of performance 
contracting, one that was most interesting and useful was the method used by 
the Baltimore Metropolitan Manpower Consortium. 

As noted in other sections of this report, Baltimore has a very large 
and complex system of subcontractors; almost all services are run by agencies 
outside the prime sponsorship. Consequently, the process of negotiating a 
contract for services is a most Important one. A great deal of staff energy 
and resources are devoted to this process, which may take place at several 
times during the year as contracts are made to meet changing circumstances. 
Intensive negotiations always take place during the planning process for the 
upcoming fiscal year. 

The Baltimore staff initially started with "goal oriented 11 contracts but 
they shifted to performance contracts in the last two years. The nature of the 
contracts varies with every deliverer because the staff has information on 
the strengths and weaknesses of each one and makes use of this information in 
negotiations. There are similarities between types of contractors, however. 
The Manpower Service Centers, which handle intake, referral and direct 
placements, are reimbursed for enrollment and for placements. They are also 
required to meet certain quality standards, such as a 3 to 1 referral ratio to 
training agencies. 
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The training agencies are reimbursed on a more complicated basis* Some 
are paid only for completion and placement. Others are paid percentages of 
the total cost of services at various stages, such as initial enrollment, 
training mid-point, completion of training, and placement. The actual contract 
depends on the staff f s assessment of the contractor's performance. For 
instance, if the staff finds that enrollees are finishing the mid-point of 
training, but not completing the program at the same levels, then they may put 
more of the reimbursement in the completion end of the program. This will give 
the subcontractor more incentive to get people to finish. 

An example of the way Baltimore uses the performance contract is 
displayed below: 



Changes 


in Performance 
(percent paid 


Contract Over Time 
to contractor) 




Enrollment 


Year 1 


Year 2 


Year 3 


85% 


30% 


30% 


Mid-Point 





30 


30 


Completion 


15 


30 


20 


Placement 





10 


20 



In the first year of the contract they placed primary emphasis on enrollment. 
In the second year they were able to push the contract to concentrate on getting 
the person through training. In the third year they added even more emphasis to 
the placement of trainees. By adjusting the amount of reimbursement for 
different tasks they are able to make the contractor do what they want them to 
do and help the contractor improve in areas in which they are relatively weak. 

Baltimore also uses an interesting method for reducing creaming in 
their programs. They negotiate entry skill levels with the subcontractors and 
attempt to keep them relatively low. They have written bonus clauses into 
the contracts of several training deliverers so that they will be encouraged to 
enroll those that fall below the negotiated entry level. For example, they pay 
several contractors a $100 bonus for enrolling and placing non-high school 
graduates . 

The integrity of the performance contracting system is maintained by the 
staff program analysts who monitor the enrollment levels and other Management 
information Reports of the contractors. Equally significant is the fact that 
completion is defined as the attainment of a specific skill level, rather than 
simply finishing the program by putting in a required amount of time. The 
skill levels are tested by the staff of the prime sponsor evaluation unit or 
by representatives of local industry. Finally, the staff verifies all 
placements independently so that they pay the contractor only for work actually 
performed. 
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G PARTICIPANT PLACEMENT STRATEGIES 

Although the acknowledged purpose of most employment and training programs 
is to prepare participants for eventual placement into unsubsidized jobs, there 
is surprisingly little solid knowledge about the most effective placement 
strategies. This problem is particularly troublesome, and almost universally 
acknowledged, in the area of private sector employment. CETA administrators 
talk a lot about "job development" and "marketing," but it is fairly clear 
that staff members who work on placement serve primarily a labor exchange 
function. They do not often convince employers to create "new" positions, 
but instead direct trained CETA clients to openings that have been identified 
through traditional sources. 

Many who have observed CETA programs are surprised at the fact that 
relatively few staff members work on job placement. This would seemingly be 
an area that would command a significant share of staff resources. However, 
most prime sponsorships devote a rather small amount of their staff resources 
to this function. One reason for this is that job placement must compete with 
many other functions and services for CETA administrative resources that 
are devoted to maintaining the inputs to the system and meeting federal 
requirements. Another is the recognition that CETA job placement is mostly a 
labor exchange activity, which in theory should be available through local 
employment security offices (ES). Prime sponsors with and without the 
encouragement of the DOL have established every imaginable type of relationship 
with local ES offices to promote job placement, but in nearly every case one 
conclusion seems to emerge: job placement for CETA clients cannot be adequately 
provided by relying only on the regular ES placement service. There is a 
legitimate need, in other words, for some sort of job placement service 
designed specifically for CETA clients. 

Since placements are a critically important outcome of CETA programs, and 
are readily quantifiable, they often become a key evaluative indicator. 
This, obviously, creates an incentive for prime sponsors, program operators 
and others to pursue a strategy of maximizing the number of placements, without 
much regard for the quality of those placements. Thus, a practice well known 
by those in the CETA system is to place people into jobs for which their 
training does them very little good, and which offer very little in the way 
of long term job stability or advancement. This type of strategy does a lot 
to undermine the value of CETA programs, and although * prime sponsors using 
it can often display impressive placement rates, they will obviously not be 
included in our presentation of exemplary placement strategies. 

The following discussion of placement strategies emphasizes the "indirect 
placement" of participants who have received some type of classroom training. 
Most CETA prime sponsors also place a certain number of applicants directly. 
That is, they succeed in finding jobs for applicants who come to them job 
ready. Direct placement strategies will not be discussed because most prime 
sponsors do not have explicit strategies for placing job ready applicants, 
and because this activity seems peripheral to the main mission of CETA. 
We do not wish to imply that only classroom training programs are relevant to 
placement; WE and PSE programs will also be discussed where they are part of 
a comprehensive effort to prepare participants for eventual placement. 
However, the identification of OJT positions, which often takes place in 
conjunction with efforts to place clients who have received training in the 
other programs, will not be addressed. 
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A final point that should be made by way of introduction is that not 
all prime sponsors have anything resembling a "placement strategy," 
Placement is often only a function that constitutes one part of a system 
of unconnected and uncoordinated functions. Taking into account all of 
these initial observations, it seems reasonable to assert that the ideal 
CETA placement system would involve two basic components: First, an 
arrangement for using job openings that ES and other placement services have 
identified, as well as the capacity for identifying and developing additional 
openings, either by canvassing employers, or by some other means; second, a 
method for matching clients to job openings so that the employers 1 needs are 
met, and so that the clients are receiving the kinds of jobs that make use of 
their CETA training, and lead to long-term employment. 

Working with the Employment Service 

The introduction to this section suggested that prime sponsors should 
establish some sort of arrangement with ES for job placement. However, many 
prime sponsors have no such arrangement, and, in many cases have sound 
reasons (usually a lack of faith in ES f ability to perform) for not having 
them. A few examples from among those which do work with ES on placing 
CETA participants merit discussion in this section of the report. 

The San Francisco prime sponsorship has a fairly typical placement 
arrangement with ES (JSO in this case) , but one that works quite well because 
of the strong working relationship between the CETA staff and the JSO, a 
relationship developed over a number of years, and the prime sponsor's 
effective use of performance contracts to insure placement results. The 
arrangement that has been established gives the JSO primary placement responsibilit 
for all graduates of the skill center and those coming out of a few of the 
smaller subcontracted programs. Most sub-contractors in the system are 
responsible for their own placement, but the JSO serves as a back-up placement 
agency for unplaced graduates. The prime sponsor pays for ES personnel who 
carry out this placement activity and for access to the Job Bank. Such a 
mixed system of placement responsibility, where many agencies are involved, has 
a lot to recommend it because both the JSO and the sub-contractors recognize 
that they must perform, since there are other agencies in the city that are 
in a position to assume their duties if they do not. 

The Baltimore Metropolitan Manpower Consortium has an interesting and 
apparently desirable, placement arrangement with ES. Several of the 21 
multi-service centers are operated by ES on a sub-contract basis. The service 
centers are involved in job placement, as are all the other MSCs, the 
training -subcontractors, and a central marketing unit of the MOMR. The ES-run 
MSCs use the Job Bank to secure placements for their participants, and also 
make the Job Bank listings available to all the other MSCs. In return the job 
openings identified by the central marketing unit and those developed but not 
filled by the other MSCs are shared with the ES offices. This is seemingly 
an ideal arrangement for a CETA system in that the Job Bank, while not being 
an exhaustive list of job openings for any area, is certainly a valuable job 
development tool, particularly when it is supplemented by job openings 
actively sought out by MSC job developers and the MOMR f s central marketing 
unit. 

Improving Private Sector Relationships 

A very effective way to expand the range of job opportunities available 
to CETA participants is to establish direct links between the CETA system and 
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employers. Many prime sponsors do this in one way or another, but a few have 
been especially effective in this type of effort. In Baltimore s the labor 
market advisory councils (see our earlier discussion of them) represent a 
very useful set of employer contacts. These contacts facilitate job 
placement activity in three ways. First, the feedback the LMACs provide on 
staff projections of demand occupations helps to ensure that only those 
training programs that have a good chance of leading to jobs are funded. 
Second, LMAC members are sometimes able to identify areas of employment demand 
that did not show up in the staff *s labor market analysis. Third, the exposure 
to the CETA system that comes with LMAC membership stimulates many participating 
employers to hire CETA clients who complete some of the training programs they 
have had a hand in designing. 

The interview boards in Syracuse, which are made up of private sector 
employers (see the section on client intake and assessment) also end up being 
an informal means of securing placements for CETA clients. The principal 
function of the interview boards is to give participants some experience 
with real interview situations, but it is not uncommon for a member of the 
interview board to offer a job to a participant with whom he/she is 
particularly impressed. The senior staff of the OFSAC are also encouraged to 
be active in the community (for example, by serving on boards), in order to 
spread the good news of CETA to as many employers as possible. 

The Omaha Consortium has always believed in making a very broad-gauged 
effort to interest members of the business community in CETA. This public 
relations activity has resulted in many jobs for CETA participants, and is 
acknowledged as having an important place in the CETA system. The 
consortium has a formal public relations director as part of the staff, and 
a PR budget. There are three main facets to the PR effort in Omaha. First 
and foremost it is directed at reaching employers and informing them about 
the mission of CETA, and the trained manpower the program has available. 
This is done in many and varied ways including mailing a newsletter to members 
of the Chamber of Commerce, media advertising, and even handing out 
lighters and keychains to businessmen downtown. The second objective of the 
PR effort is to advertise the presence of CETA to the pool of potential 
clients, and the third is simply to maintain a positive image for CETA in the 
community. Media, poster and bill-board advertising, and PR newsletters are the 
principal vehicles for accomplishing these goals. Almost everyone connected 
with the system agrees that the PR effort has been a notable success in moving 
the consortium in a positive direction in all three of these areas. 

In Albuquerque the planning staff of the OCETA worked closely with 
employers and other agencies in planning the location of the skill center. 
They decided to build the center on the site of a future industrial park, 
and then secured a grant from the Economic Development Administration to 
make physical improvements on the site in order to attract industry. The 
skill center is thus in an ideal position to train employees for industries 
locating in the industrial park, which should pay off handsomely in terms 
of future placements. 

The preceeding discussion has shown that employer contacts lead to 
placements in two basic ways: (1) By indicating promising areas in which to 
train CETA participants, and (2) by directly hiring either trained or untrained 
participants. This second activity can be taken advantage of by any prime 
sponsor. The first method, however, requires that a prime sponsor have a 
training operation that is flexible enough to respond to input from employers 
in a timely manner. This is a problem for many prime sponsors who, for various 
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reasons, find themselves to be more or less committed to a relatively narrow 
range of training programs year after year. Avoiding this kind of entrenchment 
often pays off in terms of placements. 

The Baltimore consortium has managed to keep its training operation flexible. 
Only those sub-contractors who perform are re-funded, and they always come up 
with the money to fund promising new training programs. This puts them in a 
position to benefit fully from the information they receive from the LMACs. 
The same is true of the Penobscot Consortium. They have no long-rterm 
commitments to any specific training programs, and try to design each year f s 
set of training offerings to fit their analysis of labor market conditions. 
Recently the Denver consortium has also decided to take full advantage of its 
new system of labor market analysis (see the labor market analysis section) , 
by adding a number of new agencies to their list of training subcontractors. 
These new agencies will offer training courses in the areas suggested by their 
analysis. 

Using Employability Development Plans 

The effective matching of participants to jobs is also enhanced by the 
use of employability development plans. Properly used, such plans are a 
vitally important device for ensuring that CETA participants are thoroughly 
prepared for unsubsidized employment in the area for which they have been 
trained, and have every opportunity to make a meaningful career out of 
that employment. Indeed, under ideal conditions a prime sponsorship is 
virtually in a position to guarantee jobs for clients who successfully 
complete the steps indicated in the employability plan. Obviously, CETA 
prime sponsors rarely operate under ideal conditions, but some are very 
vigilent in trying to implement the ideal model. For example, the Penobscot 
Consortium not only devotes a great deal of effort to constructing an 
assessment matrix for each participant in order to pinpoint the gap between 
each client's skill and his or her occupational goal, but they also secure 
hiring commitments for the exit components of most participants 1 employability 
plans. The Bergen County CETA staff has also enjoyed considerable success in 
using employability development plans to ensure that participants coming 
out of the system are in a position to take full advantage of the job openings 
that exist in the area for which they have been trained. (For a discussion 
of these employability development plans see the section on client intake and 
assessment.) 

Making High Quality Placements 

A final way to promote quality placements is to establish incentives 
within the system that stimulate those working on placement to make an effort 
to secure high quality placements. Few prime sponsors do so, mainly because 
the incentives created by national and regional DOL policy work in the opposite 
direction. DOL reporting requirements and concerns most evident at the local 
level have consistently ignored long-range impacts in favor of short-term 
results. 

The Baltimore consortium builds incentives for quality placements into 
their performance contracts with agencies operating training programs. A 
typical contract calls for 25% of the total funding a subcontractor is 
scheduled to receive to be paid on the basis of placements. To receive the 
full 25% the subcontractor must meet the placement quota contained in the 
contract. Moreover, placements are defined in the contracts as permanent, 
full-time, unsubsidized, and training-related jobs at or above a specified 
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wage. The MOMR staff is the sole judge of whether a job is or is not training- 
related, and the minimum wage level is set by the labor market information 
staff in conjunction with the LMACs. 

The Penobscot Consortium employs a similar approach, although with an 

in-house system, it is less formalized. The senior staff and those involved 

in placement activity have as their working definition of a placement a 

permanent, full-time, unsubsidized, training-related job, at or above a 
specified wage level that is set by the staff. 

It should be clear that establishing incentives for quality placements 
requires some sort of verification capability, and if retention is to be a 
factor in evaluating the quality of placements, a solid follow-up system 
is needed. The integrity of the verification process would seemingly be 
best protected if those responsible for verification were independent of 
the organization responsible for placement. In Baltimore all reported 
placements are independently verified by the MIS unit of the MOMR within 7 days 
after the placement has been reported. The verification process simply 
involves a check to see that the former participant is working at the wage 
level reported, and to secure a statement from the employer that the job is 
indeed permanent. There is also a regular follow-up on all terminees 30 days a 
they have left the program, and a 6 month follow-up for selected samples of 
terminees . 

San Francisco also has a sound verification and follow-up capacity. The 
JSO acts as a verification unit for all reported placements and other terminees 
The follow-up unit contacts all those who were placed, at regular intervals up 
to at least 1 year, in order to obtain data on such factors as wage gain, job 
retention, and participant satisfaction. The Penobscot Consortium also 
has a regular follow-up system that collects even more extensive information 
on all those placed out of certain special programs, and a sample of all 
others, at regular intervals for at least three years. Their long-term follow- 
for a selected sample of participants will extend for 5 years after the time of 
termination. 
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V. CONCLUSIONS 



Our most general finding is that careful planning by prime sponsorship 
staff does, indeed, have the potential for helping improve program performance. 
At the same time it is quite clear that planning is only one of a number of 
general aspects of prime sponsorship management that needs to be handled 
adroitly in order to enhance the chances of succeeding with programmatic 
goals. The importance of these general findings is underscored by the fact 
that the management variables (including planning) that can be handled in 
ways to improve CETA performance are, for the most part, quite manipulable 
by prime sponsorship staff. Elements of the local programmatic context 
such as economic conditions and demographic characteristics of participants 
are not highly constraining on the kind of performance that can emerge. This 
means that even under economic conditions that conventional wisdom holds to 
be adverse to good CETA performance such high quality performance can, in fact, 
emerge. It also means that prime sponsorships can target their resources on 
the most disadvantaged part of the eligible population as measured by gross 
demographic characteristics without diminishing their potential for good program 
performance. 

This concluding section is kept short both to highlight what we think is 
most important in what we have found and also because detailed findings are 
reported in ample detail throughout the body of the report. The first section 
highlights some of the principal empirical findings. The second section makes 
some broader observations. 



A. MAJOR FINDINGS 

The Effects of Context on Planning Systems 

General contextual factors are not determinative of what kind of planning 
systems emerge in individual prime sponsorships. Prime sponsorships can design 
the kind of planning system they choose for programmatic reasons and are not 
forced into a specific planning mode by external factors. 

The general management context in a prime sponsorship is, however, related 
to the kind of planning system that seems most likely to emerge. This is 
probably because certain styles of planning fit best with certain styles of 
management. In a sense, management decisions "cause" the kind of planning 
system; but in another sense planning emerges along with other features of 
management and the whole package tends to fall into predictable patterns. 
Thus, "association" is probably a better way of describing the relationship 
between management context and planning systems than "causality." 

In prime sponsorships we identified as having future-oriented planning 
systems the associated management variables included a pattern of staff -dominated 
decision-making, a stable and high quality staff, stronger than average support 
for staff from political officials, thorough monitoring, and the active 
involvement of the private business sector. 

In prime sponsorships with operations management planning systems there 
was more likely to be less staff domination of decision-making (usually 
involving significant sharing of influence with service deliverers) , more 
turnover in key staff, low involvement of business, and less well-developed 
monitoring systems. 
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Explaining Program Performance 

Management characteristics and planning both have stronger influence on 
performance than any contextual variables. This is a very encouraging finding 
in that this means that prime sponsorships particularly the staff have the 
latitude to focus on management decisions and planning, over which they have 
considerable control, in order to improve performance. Conventional wisdom 
that suggests that economic conditions and demographic characteristics of 
participants determine or at least highly constrain performance simply finds 
no empirical support in our work* 

The factors we found to be most strongly associated with better performance 
were the quality of staff, the nature and extent of business involvement, and 
the quality of monitoring. Prime sponsorships not satisfied with their 
program performance would be well-advised to consider their activities in 
relation to these three areas first. The body of the report contains some 
specific suggestions of steps that might be taken in each of the areas to 
improve conditions that would, in turn, have a salutary effect on performance. 

Exemplary Approaches to Critical Elements of Planning and Management 

Given that a variety of management and planning factors were found to 
affect program performance we thought it worthwhile to spell out in 
some detail specific instances of approaches to some key elements we had observed 
to be working well. It needs to be underscored that we do not find a single 
"good" way to approach any of these elements. Rather we found a number of 
seemingly productive approaches. And, with adaptations appropriate to 
specific local necessities and conditions, we think some t>f these approaches 
deserve publicity because they may well merit emulation in spirit if not in 
every detail. Diffusion of productive approaches within tne CETA system seems 
both possible and eminently desirable to us. 

We provided what we considered to be good examples of productive 
approaches to seven elements of planning and management: 

1. Manpower planning councils. 

2. Monitoring and evaluation. 

3. Universe of need and target group identification. 

4. Intake and assessment strategies. 

5. Labor market analysis. 

6. System design (service deliverer selection, service delivery systems, 
and performance contracting) . 

7. Participant placement strategies. 

Our "findings 11 in this area cannot be readily summarized except to say 
that we found multiple examples of productive activity in each area. The 
text of Section IV contains considerable detail. 



B. OBSERVATIONS AND IMPLICATIONS 

Is There One "Best" Model of Planning? 

We have identified three models of planning in this report: Future-Oriented, 
Operations Management, and Crisis Management. By implication there is a fourth 
model: No Consistent Planning. We think there are high programmatic costs for 
a prime sponsorship that adopts the No Consistent Planning Model. We also see high 
costs to deliberate pursuit of a Crisis Management Model or, more to the 
point, continued toleration of conditions that lead to the necessity of adopting 
a Crisis Management Model. But we do not argue that Future-Oriented planning 
is necessarily superior to Operations Management planning in all cases at all 
times. 

Clearly, the Crisis Management model characterizes a system that all 
prime sponsorships should try to avoid. Unstable influence patterns, unmanaged 
conflict, and inoperative feedback processes are all marks of a system in 
chaos that can have little hope of fulfilling the substantive goals of a 
CETA program. However, while prime sponsors would certainly rather be in an Opera- 
tions Management or a Future Oriented system, it is not "the case that the Future 
Oriented model is preferable to the Operations Management model in all cases, 
contexts, and times. For example, at a certain point in a prime sponsor's 
development, it may be that most long range goals have been sufficiently 
dealt with so that the continued extension of organizational resources into an 
ongoing Future Oriented part of the staff may be inefficient. The prime 
sponsorship may make the best use of resources by focusing just on ongoing 
programs within annual plans. At some later point it may be efficient and 
important to add the features of Future Orientation to the existing accomplishments 
of the Management Operations model, hold the Future Orientation in place long 
enough to inject additional planned change into the system and institutionalize 
it, and then revert to the Operations Management of the now institutionalized 
gains. Thus, we believe the two models accomplish somewhat different 
purposes, and a prime sponsorship staff can consciously shift between the 
two models of planning according to local needs and priorities. 

There are not only one or two "good" sets of planning activities (and 
related influence structures) that should be adopted in all prime sponsorships 
in all contexts. Some types of planning activities and some types of influence 
structures will be effective in some contexts while not in others. 

For example, a very sophisticated, highly quantitative monitoring system 
may be essential for good planning and decision-making in a large prime 
sponsorship with numerous service deliverers. Yet, in a much smaller prime 
sponsorship, one with only one or two service deliverers, a monitoring system 
based largely on qualitative information, on a less rigorous basis may work 
quite well and resources may, in fact, be wasted on a more sophisticated 
system. However, if no monitoring system exists and there is no feedback to 
decision-makers on how well programs are operating, then a judgment can be 
made that the planning system is not functioning effectively. And to the 
extent that planning is Important for good decision-snaking, then the decision- 
making process could also be judged to be poor. 

What Gets Planned? 

We found that four major areas are the focus of planning efforts: (1) target 
groups in the population, (2) target occupations, (3) program mix, and (4) service 
deliverer identity and responsibilities. 
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Most of our sites were quite self-conscious about planning for selection 
of their target groups. Where there was an active public process it was 
invariably used on this question. Different methodologies were used for 
identifying target groups. The wisest strategy seems to be to keep the 
definitions of target groups broad rather than using elaborate matrices that 
result in very small discrete categories. Broadly inclusive categories seem 
easier to work with both operationally and politically. 

There was considerable attention to occupational planning in about 
half of our sites. These prime sponsorships did a thorough and impressive 
job of working with all kinds of data on target occupations. Spending lots of 
time and resources in this area seems to make particularly good sense if the 
system is flexible on a short-run basis, so that it can respond to short-run 
occupational opportunities that are identified. If the structure of the system 
and/or political reality prevents much short-run change, then less attention to 
target occupations may well be warranted because it might turn out to be 
primarily an academic enterprise. 

Where there are subcontractors, program mix and the choice of service 
deliverers are usually planned or addressed together, with each being 
a function of the other in a chicken-and-egg relationship. Where much or all of 
the delivery is in-house or where a good deal of individual referral is done 
or where subcontractors are closely managed, then there was a greater tendency 
to think of program mix questions prior to worrying about the details of 
service deliverer identification. But, particularly where there are strong 
community-based organizations with important local political support, the 
identification of service deliverers and, to some extent, what they deliver 
is almost a given. 

It is also worth noting that all planning systems we have observed focus 
virtually exclusively on CETA matters. Even in the future-oriented sites we 
did not observe systematic integration of CETA planning with community develop- 
ment planning. In short, CETA planning focuses only on planning activities 
with relation to the supply side of the labor market; planning linked to 
non-CETA programs with concrete actions on the demand side of the labor market 
is still largely virgin territory. 

What Promotes and Impedes Effective Planning? 

Five major conditions promote effective planning if present and impede 
effective planning if absent. 

First, goals are essential. These need to be self-consciously addressed 
and articulated. And this process needs to be repeated; it cannot be assumed tl 
goals will be internalized if announced only once. Lack of clearly articulated 
goals or the presence of goals that are almost all implicit can create problems 
in both planning and, ultimately, in performance. 

Second, some form of public planning process or allowance for major 
input from actors outside the staff is very useful in promoting planning 
decisions that are responsive to conditions in the "real world" and also in 
promoting decisions that are widely accepted as legitimate by the major actors 
in the system. Most of our sites have chosen to make their advisory council the 
centerpiece in a public planning process. 

Third, both good monitoring and well-developed Management Information 
Systems (MIS) are essential for linking planning to operations and using 
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both in concert to improve performance. Evaluation is more rare, but where it 
exists it is a useful addition to the basic monitoring and MIS activities. 

Fourth, and perhaps most important in terms of being a prelude condition 
for many of the others, prime sponsorships need to be very self-conscious in 
developing a solid staff ample in number, well-trained, willing to stay with 
the job at hand for a period of some years, and committed both to making the 
program succeed in general and to some concrete programmatic goals. 

Fifth, the CETA staff needs to have a productive relationship with its 
subcontractors. Where those subcontractors are strong performers with a strong 
independent political base this relationship may well have to be based on 
negotiation and compromise aimed at creating a "family feeling" about the 
whole system. 

The continuing categorical expansion of CETA has put up some roadblocks to 
planning, especially in terms of integrating different CETA programs into a 
comprehensive whole. However, a high level of determination to achieve 
integration can still have a high degree of payoff. 

How Important is Planning to Performance? 

We have found some aspects of planning to be related systematically to 
some aspects of performance. These concrete findings should not be overblown 
into a claim that planning is the key to good performance. There are many 
other elements of management that are also critical in developing the capacity 
for good performance in a local CETA system. Planning is not a panacea; it is 
important. Fortunately, there are concrete ways to strengthen planning it is 
not just a matter of engendering positive attitudes toward it on the part of 
a staff. The central point of Section IV of the report was to specify some 
of the concrete options available to prime sponsorships in a number of both 
planning and broader management activities. 

In the broadest sense, the fact that manipulable aspects of planning and 
management have been found both in this study and in our earlier work (Ripley 
and associates, 1978) to be related to the quality of performance is highly 
encouraging. There is nothing magic about decentralized employment and training 
programs that makes them work better than other kinds of programs. But there 
is solid empirical evidence that local prime sponsorships can make specific 
choices that will have desired programmatic effects. They are not caught in 
a mechanistic situation in which their efforts are automatically overridden 
by fluctuations in matters over which they have no control, such as the general 
economy. 
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APPENDIX: BRIEF DESCRIPTIONS OF THE STUDY SITES 



Note that these descriptions apply to the time of our field visits 
either spring or summer, 1978 (see Table 2 in the text). The sites are in 
alphabetical order. 

ALBUQUERQUE-BEKNALILLO COUNTY CONSORTIUM, NEW MEXICO 

The Albuquerque prime sponsorship is a consortium involving the City and 
the County in which it is located. The City, which contains over 70% of the 
population in the County, has been given the responsibility of administering 
the CETA Program. 

Since 1976 Albuquerque's unemployment has been higher than the national 
average. The City has been paralleling the national employment picture recently 
by experiencing a significant drop in its unemployment rate since it peaked 
during the second and third quarters of FY 1977. 

Albuquerque is an area of growing population with a large number (30% 
according to the 1970 census) of Spanish surnamed citizens. The census 
indicates that black population is suspected of having increased its proportion 
since then. More recent data indicate that Spanish surnamed individuals 
comprise 44% of those unemployed, while blacks and American Indians are 4% 
each. Ethnic groups have not been designated as significant segments but are 
served in accordance to their proportion of the economically disadvantaged 
universe of need. The significant segments that have been identified are: 
elderly (45 and over), handicapped, offenders, and youth. 

Staff has been dominant in determining the basic features of the Albuquerque 
Title I program. In large part, this may be attributed to the fact that the 
staff is composed of an unusually large proportion of individuals who are 
clearly manpower professionals and whose experience pre-dates CETA. In part, 
staff dominance can also be explained by the lack of any effective opposition 
to them from either elected officials, the Planning Board, or service deliverers. 

There have been only slight changes in the Title I program (participants, 
program mix, and service deliverers) over the past few years. This situation 
clearly reflects the staff goal of stabilizing the program and building up 
the capability of existing deliverers. It also reflects the : fact that, 
according to the standard indicators, Albuquerque has been running a very 
successful program and so there has been little incentive to institute major 
changes. Operating a good program, one that performs well given such standard 
indicators as placement rate, was identified as the most important goal held 
by the Albuquerque CETA program. 

The typical Title I planning that takes place in Albuquerque consists 
of incremental adjustments made in response to operating problems that have 
been identified. Albuquerque was engaged in some long term planning in the 
past, but none can be said to be taking place now, although re-establishing this 
capability is clearly a goal of the new CETA Director as is revitalizing what 
has been a staff dominated Planning Board and thereby opening up the planning 
process to greater public input. 



As indicated, Albuquerque has done well on the standard program perfoi 
measures such as planning rate, non-positive termination rate, and cost pei 
placement. The recent redefinition of an indirect placement has caused 
performance on the indirect placement rate and cost per placement indicate* 
to drop during the second quarter of TT 1978 but the prime sponsor has take 
corrective action and hopes to finish the year meeting its goals in these 
areas. High goal achievement has been achieved without sacrificing service 
to those participants determined to be most in need. 
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ATLANTA, GEORGIA 

The City of Atlanta prime sponsorship exists in a region dominated by 
the wholesale and retain trades where 60% of those employed hold white collar 
jobs Atlanta has suffered a higher unemployment rate than its neighboring 
suburbs and counties. The unemployment rate for Atlanta has also been 
higjher than the national average. 

Atlanta has been losing its white, middle class population to the 
suburbs. (Jobs, too, have been moving in this direction.) In 1960, for 
example, blacks were 39% of the City f s population. By 1975 it is estimated 
that this figure was 60%. As might be expected, Atlanta's blacks have a 
much higjier unemployment rate than the white population and comprise 81% 
of those unemployed. 

The most outstanding feature of the Atlanta prime sponsorship is the 
openness of its decision-making process to the input of a wide range of 
individuals and organizations. Decisions are typically made on the basis of 
what is perceived to be the best available information. Influence is shared 
among the staff, the advisory council, service deliverers, and the Mayor, 
although usually the staff position is dominant. 

The CETA office staff itself is generally competent and well motivated. 
However, there have been important problems with Title I MIS and with the 
Fiscal Unit, and the Evaluation and Planning Units have been understaffed. 
Improvements are being made or are planned for all of these areas. 

Only slight variation has occurred since FY 1975 in the allocation of 
resources and participants among program components. The most noticeable 
exception appears to be decline in the proportion of clients enrolled in 
classroom training and an increase in work experience. Most major contractors 
(chosen annually via a Request for Quotations process) were picked up from 
the pre-CETA days and have continued to be ref landed. More change has occurred 
among the smaller deliverers. 

Long-term planning cannot be said to exist in Atlanta. The planning that 
does take place is short-term and is focused on the Title I grant application 
process and the selection of service deliverers. Service deliverer selection, 
which actively involves the advisory council and staff, usually also determines 
target groups, target occupations, and program mix, although the advisory 
council does independently look into these areas. Incremental rather than 
radical change has been the result of the planning process used in Atlanta. 
Often, this change has been stimulated by reaction to poor program performance, 
especially lack of placement, by the service deliverers. The monitoring and 
evaluation efforts (especially the former) of the CETA Office have been instru- 
mental in identifying many of the problems that have led to change. 

Atlanta's placement rate has typically been above the national average, 
just as its cost per placement has been lower. Non-positive terminations, 
on the other hand, have been above the national average. Atlanta's planned 
versus actual figures have been generally about the same with the exception 
of the non-positive termination rate and expenditures, which have lagged 
behind estimates to the point where a $1.25 to $1.5 million underspending 
problem developed. 

Locally established goals have been pursued successfully with the goals 
of infusing more innovation into the system and developing a more integrated 
manpower delivery system being the exceptions. 

307-746 0-80-9 ^ 



Atlanta has tended to serve its chosen target groups, but the choice 
of groups itself has tended to reflect previous, successful service patterns 
The typical client served in Atlanta is black, unemployed, and economically 
disadvantaged . 
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BALTIMORE METROPOLITAN MANPOWER CONSORTIUM, MARYLAND 

The Baltimore Metropolitan Manpower Consortium (BMMC) consists of the 
city of Baltimore and Anne Arundel, Baltimore, Carroll, Harford, and Howard 
counties . The consortium coincides with the Baltimore SMSA, which includes 
a population of nearly 2.2 million. Almost 70% of the population in the 
consortium resides in the city of Baltimore and Baltimore County. Twenty-six 
percent of the population in the consortium is non-white, but 53% of Baltimore 
city's population is non-white. A majority (60%) of the consortium 1 s poor is 
located in the city of Baltimore. The overall rate of unemployment in the 
consortium has remained fairly close to the national average, but the rate of 
unemployment for non-white youth has been quite high (averaging over 17%) . 

The BMMC has benefited greatly from having the principal features of its 
delivery system in place before CETA was enacted. The program is administered 
by the Mayor's Office of Manpower Resources (MOMR) , and many members of the 
senior staff of the MOMR worked together under CAMPS to design a comprehensive 
delivery system for the consortium. Experience and stability in the senior 
staff has been very important in the development of predictable and stable 
relationships among manpower actors, and in the recruitment of other sophisto- 
cated manpower professionals into the MOMR staff. The combination of an 
experienced staff and a stable delivery system has a lot to do with the success 
enjoyed by the BMMC under CETA. 

With an effective delivery system in place, planning in the BMMC has taken 
on an operational orientation. While all four types of employment and training 
programs (WE, CT, OJT and PSE) are operated under Title I, the bulk of the 
planning activity focuses on the training programs. Planning for skill training 
programs is greatly aided by a network of labor market advisory councils (LMACs) 
that have been established to supplement the information generated by the staff 
on prime sponsor training needs and performance. The skill training programs 
are subcontracted to a variety of public and private training organizations, 
while all other programs are operated in-house. 

Most CETA services are made available to clients through a series of 
manpower service centers strategically located to serve the entire consortium 
area. Job development activity is shared by a central marketing unit, job 
developers located in the service centers, and job developers assigned to the 
training subcontractors. All placements are verified by the MIS unit of the 
MOMR. The MOMR uses a system of performance contracting to guarantee that 
all subcontractors are meeting the performance goals established each year 
through negotiations between the staff and the subcontractors. Quality 
performance is further insured by an extensive monitoring and evaluation 
effort, which is backed up by a solid MIS. The evaluation unit is also 
continuously studying various aspects of program operation and performance 
by conducting both short term studies of program outcomes, and long term 
follow-up evaluations of CETA programs . 

The standard program performance indicators demonstrate that the BMMC is 
operating a strong program. Placement rates have been consistently above the 
national average, and non-positive termination rates and costs per placement 
have steadily declined. This has been accomplished without moving away from 
serving disadvantaged, which" has always been a primary goal of the BMMC. The 
staff believes that this record of performance is linked to their continuous 
efforts to improve program operations through systematic planning. 
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BERGEN COUNTY , NEW JERSEY 

The Bergen County, New Jersey, prime sponsorship has a population of 
around 900,000 people. Most of them (97%) are white and relatively wealthy; 
the county is among the wealthiest in the nation. But there are obvious 
pockets of poverty spread through this large county separated from New York 
City by the Hudson river. The county f s unemployment rate has often 
been higher than the national average over the last three years. 

The single most important feature of the prime sponsor's organization is 
that the entire CETA program is operated by the Bergen County Community Action 
Program (BCCAP) under a contract with the County's Board of Freeholders an 
arrangement made at the beginning of CETA. The BCCAP staff operates most 
CETA programs in-house through three multi-service centers where clients 
receive intake, assessment, counseling, work experience, job placement, and 
job development services. Only classroom training for skills or educational 
improvement is subcontracted. The system of in-house operation, a central 
goal of the prime sponsor staff since the beginning of CETA, has been 
largely accomplished. 

Primary emphasis in the Title I program is on training programs in the 
classroom or through OJT. Limited resources are devoted to work experience 
programs. Over time the preference for classroom training and the aversion 
for work experience has become manifest in larger commitments of program 
funds to the former and less for the latter. 

Bergen County's staff has an operationally-oriented view of planning. 
The planning unit places its emphasis on continuous, cooperative work with 
the operations staff to improve program performance. Its principle strengths 
are in the areas of system wide development and program component monitoring. 

The MIS has been weak because of continuing problems with a faulty 
computerized system. The poor MIS has prohibited the staff from conducting 
program-outcome evaluations, but client satisfaction questionnaires, on-site 
reviews, and other forms of monitoring are regularly conducted by the staff 
and the advisory council and are utilized in program decisions. 

The standard program performance indicators demonstrate that Bergen is 
operating a steadily improving program, the results of which are typically 
better than the national average. Across the three years of complete operatic! 
the placement rate has increased 18%, the non-positive termination rate has 
improved by 16%, and the cost per placement rate has improved 36%. In 
general it would seem that this has been accomplished without Creaming for 
the best applicants. The staff feels that these improvements are linked to 
their continuous efforts to improve program quality and operations througji 
systematic planning. 
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DENVER, COLORADO 

The Denver prime sponsorship encompasses only the city and county of 
Denver (they are co-terminous) . It has about one-third of the population 
of a larger labor market and a diversified economy that is not reliant on 
heavy industry and manufacturing. Denver's unemployment rate, although higher 
than surrounding suburban counties, is lower than the national average. 

The population is composed primarily of Anglos, with two large minorities 
Chicanos (17%) and blacks (9%). CETA significant segments include Chicanes, 
blacks, economically d is advantaged, women, heads of household, and veterans. 

The Denver Manpower Administration (DMA) staff is large and has grown 
very rapidly since mid-1976. The staff has been wracked by turnover, especially 
at top levels, and by long- term vacancies. The planning staff has grown from 
one consultant in July, 1977, to seven professionals in April, 1978. Except 
for the chief planner, the staff is relatively inexperienced. The MIS unit 
in DMA has been particularly plagued with problems that have prevented the 
unit from functioning. 

There has been little substantive change in Title I manpower programs 
in Denver since pre-CETA days, although there have been some procedural 
changes, most notably the introduction of an RFP for the selection of 
service deliverers for the FY 77 program year. Program mix has been relatively 
stable over time and emphasizes counseling and assessment services for 
clients and, to a lesser extent, classroom training (including adult basic 
education). There has been little OJT or work experience. The identity of 
the major service deliverers has remained constant, although a few new ones 
have been added since the introduction of the RFP. 

The amount and quality of planning for FY 78 were severely limited by 
operational problems that demanded the time and attention of the planners. 
Planning activities that have been relatively strong are 1) the analysis 
of the universe of need and target group selection and 2) the analysis of 
occupational opportunities. Planning has been especially weak in relation 
to decisions about program mix, which has emerged simply as a function of the 
deliverers selection. Planning has also been severely hampered because very 
little monitoring has taken place for over a year. Because of the lack of 
monitoring, MIS, and time no evaluations of programs have been done. The 
involvement of the. advisory council in planning has been very limited and 
was under close control of the DMA Administrator until FY 78. At present the 
planning staff is trying to support the development of an active and 
influential council. 

Denver ? s program performance has been strong in the area of reaching 
target groups, which contain those individuals most in need. However, there 
are a number of weak areas in Denver 1 s performance: placement has slipped 
recently, even though most placements are direct; the non-positive termination 
rate has remained fairly high; and very few participants are being served with 
vocationally-related training in a system that is currently spending over 
$3.5 million annually in Title I. Local goals other than organizational 
survival are not stated clearly and widely understood and shared. Assessment 
of the quality of program performance is impossible until the MIS system is 
functioning reliably. 



GULF COAST MANPOWER CONSORTIUM, TEXAS 

The Gulf Coast Manpower Consortium (GCMC) is comprised of eleven 
counties: Austin, Brazoria (the eligible prime), Chambers, Colorado, Fort 
Bend, Liberty, Matagorda, Montgomery, Walker, Waller and Wharton. All of 
the counties are members of the Hous ton-Galves ton Area Council (H-GAC) , 
which serves as the administrative arm of the consortium. The consortium 
was formed in FY 76; prior to that time the counties were with the Balance of 
State operation. 

The area f s economy, greatly influenced by the vigorous growth in Houston, 
is robust. The unemployment rate in the consortium has been under the 
national average by a substantial margin. The forecast is for continued 
growth. Manufacturing and construction are scheduled to experience 
substantial expansion. 

The GCMC has two large minority groups, Blacks and Chicanos. They 
comprise 17% and 11% of the population, respectively. CETA significant 
segments include disadvantaged youth, older workers, veterans, handicapped, 
minorities, female heads of households, and working poor. 

There has been very little change in the Title I manpower program since 
CETA's inception. Work experience, especially directed toward youth, 
classroom training^and a small OJT program comprise the Title I program. 
While the substance of the program has not undergone much change, the mode 
of service delivery has changed. In the consortium 1 s first year of operation, 
each service deliverer performed its own intake, assessment, job development 
and placement operations. Those activities were subsequently given to the 
Texas Employment Commission (TEC). CETA funds a number of Manpower Service 
Centers throughout the consortium, staffed by TEC personnel, to perform 
these functions. 

A Request for Proposal is used for Title I. The GCMC is devoid of many 
potential deliverers of CETA services due to its rural orientation. As a 
result, the same deliverers have been a part of the manpower delivery system 
since the beginning. Only one deliverer, SER, has gained entrance into the 
system since that time. 

The planning for Title I is reliable in nature. The shortage of reliable 
data for some counties in the consortium precludes a planning system based 
on highly quantifiable inputs. The staff supplements the data that are 
available with a thorough knowledge of the service area. Field representatives 
visit service deliverers frequently; this type of informal monitoring compensat 
for the fact that only one program monitor is employed by the GCMC. Planning 
has been traditionally a staff function with only limited input from the 
Manpower Advisory Committee. There are signs that the Committee is becoming 
more active. 

GCMC's program performance has been quite good. Placement rates have 
surpassed the national average. The non-positive placement rate is very low 
as a result of aggressive efforts on the part of counselors to keep track 
of participants who have dropped out of courses . Often these individuals 
find employment on their own and are not counted as non-positive terminations. 
The prime sponsorship has a commitment to serve the disadvantaged; 75% of 
Title I participants are mandated to be economically disadvantaged. 
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HEARTLAND MANPOWER CONSORTIUM, FLORIDA 

The Heartland Manpower Consortium (HMC) is composed of five contiguous 
counties located in the west central portion of the state of Florida: Polk, 
Highlands, Hardee, DeSotOj and Okeechobee. The population of HMC is 
approximately 400,000, of which 74% reside in Polk County. Highlands ranks 
second with 11% of the population and the other three counties each comprise 
5%. HMC f s quarterly unemployment rate is consistently higher than the 
national average. Significant seasonal unemployment is the most distinctive 
feature of the economy; the predominance of the citrus industry is responsible 
for the fluctuations. Citrus production and phosphate mining constitute the 
major industries in the area. 

The racial mix of HMC is 83% white, 17% non-white. Blacks are the major 
non-white group in the area, with Spanish-Americans and American Indians 
comprising a small percentage of the total population. Age is a significant 
factor in the composition of HMC f s population. Persons 65 and over constitute 
16% of the area's residents, reflecting the tendency for retired persons to 
settle in warm southern climates. Older workers are a significant segment 
identified by the prime sponsor. Youth, minorities, veterans, and migrants 
are also considered to be in need of CETA services. 

HMC f s Title I program has not experienced much substantive change since the 
program's beginning. The three major service deliverers remain in place; 
they have been included in the delivery system since the program's inception. 
Some service deliverers have been defunded, primarily those involved in pilot 
projects. Significant changes in the program are forecast if the staff acts 
on its stated intention of reducing its reliance on sub-contractors and 
operating the program in-house. 

Work experience has been the dominant Title I activity. An In-School 
program and a very popular Seniors-At-Work program comprise the bulk of work 
experience activities. Classroom training, now receiving 40% of Title I 
funds, is often used in conjunction with work experience. Participants are 
enrolled in work experience after completing classroom training to enhance 
their chances of obtaining unsubsidized employment. 

The public planning process is as important as technical planning in the 
Heartland Manpower Consortium. The imaginative use of the Manpower Planning 
Advisory Council and its subcommittees is the heart of the public planning 
process. A flexible subcommittee structure enables the staff to obtain a 
wide variety of input from individuals with expertise in a manpower relevant 
field. The subcommittees are as follow: Needs Assessment of Clients, Youth 
Council, Proposal Review and Evaluation, Vocational Training, and Public 
Service Employment. The MPAC, whose chairman is an important elected official, 
works closely with the staff in all planning stages. 

The prime sponsor's performance has been improving. The placement rate 
has steadily increased. The non-positive termination rate has fluctuated 
widely but has been showing some improvement. The prime sponsor's cost per 
placement compares quite favorably with the national average. Service to 
the economically disadvantaged has been on the increase. Females have been 
steadily increasing as a percentage of Title I participants, while Title I 
service to non-whites has been decreasing. 



KING-SNOHQMISH MANPOWER CONSORTIUM, WASHINGTON 

The KSMC prime sponsorship encompasses a two-county area. Seven cities 
in addition to the two counties are members of the consortium. The consortium 
is also a single SMSA and can be considered as a single labor market, although 
there are many smaller labor markets within the area. The labor force is 
highly educated, highly skilled, and highly unionized. The economy is diversifie< 
Unemployment, which had been well above the national average, has dropped rapidly 
and the economic outlook for the area for the immediate future is relatively 
bright. 

The population of about 1.4 million people is primarily white. In the 
labor force 3.4% are black, 2.8% are Asian, and 2.0% are Spanish American. 
The minorities receive a high degree of service from CETA. Current significant 
segments are limited to three broad categories: minorities, females, and youth. 

The KSMC staff has about doubled in size to 70 in the last two years. 
In general, it is an able, experienced, and well-educated staff. Turnover in 
the last year has been relatively high. This turnover has included three 
division directors as well as the overall Director. The new Director took 
office in late 1977, replacing the former Director who had left in September, 
1977. The Planning and Analysis unit has 12 positions and works very closely 
with the Program Design and Administration unit, which focuses on operations 
and has 14 positions. 

The Title I manpower delivery system has been relatively stable throughout 
the CETA period. The major deliverers have remained the same although some 
marginal and small deliverers have been replaced. An RFP process is rarely used. 
Given at least moderate performance it is assumed that funded deliverers will 
be refunded. Over time the classroom training component of the program in terms 
of both dollars and participants has increased and the work experience component 
has shrunk. 

The staff thinks of planning in two senses: short-run, operationally 
oriented planning (that also involves grant writing and maintenance) and long- 
run, strategic planning. Most effort has been expended on the former 
enterprise and the latter enterprise has been given relatively little attention, 
in part because of the necessities of dealing with a manpower system that is 
rapidly being recategorized from Washington and in part because of lack of 
data sophisticated enough to meet the planning staff's high standards. The 
presence of a relatively new Director, whose own values and preferences are not 
yet clear to staff or other actors, also inhibits long-range planning. 

Both planning in the short-run sense and the use of a widely accepted 
public planning process are well developed in KSMC. The process involves a 
number of actors: service deliverers, subcommittees of the advisory councils, 
two advisory councils (one for each county), the staff, a subexecutive committee, 
and the Executive Board (elected officials) of the contortion. This process 
serves to diffuse information about CETA, get some feedback from the various 
segments of the community, and make virtually all decisions widely accepted 
as legitimate. There are some uncertainties in its use for FY 79 because there 
are a number of new persons in critical positions. Formal planning for FY 79 
has been postponed because of the unknown funding level for FY 79 and new 
provisions likely to be included in CETA reauthorization that cannot be fully 
predicted . 



The planning by staff has been particularly strong in terms of identifying 
target groups. The capability of the staff to affect performance is enhanced 
by the existence of a strong monitoring effort and the close relations between 
the planning and operations staff. Goals are mostly implicit and the new 
Director has not yet altered or added to those goals publicly, although he 
favors expanding CETA non-CETA linkages, especially with economic development. 
The choice of service deliverers helps determine service mix, although the 
general commitment to classroom training and OJT helps determine the choice of 
deliverers. There is relatively little systematic attention given to identifying 
job opportunities by the KSMC staff job development and placement is given to 
individual deliverers. 

Program performance has been improving and has been quite strong in the 
last several years. Long-standing local goals (although in part implicit) are 
being achieved. These include maintaining the consortium, maintaining the core 
of the existing delivery system, and focusing on placement of the most disad- 
vantaged. Minorities, females, and the disadvantaged receive heavy attention 
in the services that are delivered. Placement has been increasing, although it 
is still not dramatically high. A previously high non-positive termination rate 
has been reduced substantially and is now well below the national average. 



OMAHA COMBINATION OF GOVERNMENTS CONSORTIUM , NEBRASKA 

The city of Omaha Combination of Governments consortium is comprised 
of the city of Omaha, the balance of Douglas county, the city of Bellevue and 
the remainder of Sarpy county. Population in the consortium is estimated to 
be 585,000. The city of Omaha, the largest metropolitan center in Nebraska, 
and the remainder of Douglas county constitute over 90 percent of the 
consortium's population. The two-county area of Douglas and Sarpy county has 
a racial composition of 90.5 percent White, 8.1 percent Black and 1.4 percent 
other minorities. 

The Omaha economy is an economy in transition. The emphasis is shifting 
away from exclusive reliance on manufacturing to include service-oriented 
industries. While the transition has prompted significant upheavals, the consor 
unemployment rate did not exceed the national average. Moreover, the Omaha 
economy staged a quicker comeback from the recession than many prime sponsorship 

Omaha has long been the host to manpower programs * A CEP was in operation 
prior to CETA ? s inception. Omaha was one of the three original prime sponsorshi 
selected for pilot projects in the Comprehensive Manpower Program (CMP), the 
forerunner of CETA. The existence of these manpower training programs has aided 
the prime sponsorship in a number of ways. They created a reservoir of exper- 
ienced manpower specialists, many of whom are still involved in CETA. The 
heads tart provided by the CMP status enabled the prime sponsorship to wage a 
number of consolidation battles early in the life of the program. While 
inexperienced staff in other prime sponsorships were in the initial trial and 
error design phase, the Omaha staff knew the capabilities and limitations of its 
manpower delivery system. 

The consortium's Title I program has experienced substantial change since 
the program's beginning. In general, changes have been in the direction of 
centralizing services previously sub-contracted to other deliverers. Intake 
has been centralized; the prime sponsorship does much of its own assessment, 
orientation, education, job development and placement activities. Those functic 
primarily training, performed by service deliverers are closely monitored. 
Contracts written with service deliverers contain performance standards and 
are of a cost-reimbursable nature. 

A highly quantifiable planning system exists in the consortium alongside 
a more intuitive model. As the latter is used at the highest level of the 
organization, it gives shape and content to planning steps performed by the 
Planning unit. The philosophy of the strong Director and the rigorous approach 
of the Planning unit mesh to ensure that both political and more technical 
manpower planning are represented in the consortium. To date, the Manpower 
Planning Council has not been an important partner in planning but there are 
indications that the involvement of the Council is increasing. 

The prime sponsor's placement rate has shown steady improvement. It 
compares favorably with the national average. Due to the open intake policy, 
the non-positive termination rate has been higher than the staff would like 
but it has shown improvement in recent quarters. The prime sponsor's cost 
per placement generally has been below the national average. 
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fENOBSCOT CONSORTIUM, MAINE 

The Penobscot Consortium includes Penobscot and Hancock counties in the 
state of Maine, and operates under the authority of an Executive Board composed 
of the six county commissioners from these two counties. The population of 
the consortium is just under 180,000, and most of the population is spread out 
among several small towns in the two county area. The major population center 
in the consortium is the city of Bangor, which has a population of 33,000. The 
population is predominately white (99%) , and 15% of the people in the area are 
living below the poverty level. The consortium's unemployment rate has generally 
remained close to the national average, but shows seasonal fluctuation reaching 
levels significantly above the national average during the winter months. Paper 
manufacturing is the major industry in the area, but most of the private sector 
employment opportunities for CETA clients are in relatively small manufacturing 
enterprises . 

CETA programs in Penobscot County are administered by the Penobscot 
Consortium Training and Employment Administration (PCTEA) . The prime sponsorship 
was established in FY 1976 after a year of CETA experience under the administration 
of the state. During the first year of PCTEA control a comprehensive delivery 
system was developed, which involved the termination of subcontracts with most 
of the existing program operators, and the redirection of the thrust of 
employment and training programs away from work experience and toward classroom 
training and OJT. This was accomplished through the efforts of an imaginative 
and energetic staff with the solid support of the county commissioners. 

The comprehensive delivery system that was established in the Penobscot 
Consortium centers around three Offices of Training and Employment Programs (OTEPs), 
which were located in a manner that insured service to the entire consortium. 
These offices can make available to CETA clients the complete range of 
employment and training services because programs from all the principal CETA 
titles are integrated in a unified service delivery effort. The OTEP offices 
are coordinated by the PCTEA central staff, which is also responsible for 
planning, monitoring, and evaluating all CETA programs in the consortium. 
All the programs are run in-house with the exception of OJT development in unionized 
occupations, which is subcontracted to the AFL-CIO. The in-house system 
facilitates frequent and constructive communication exchanges among all 
staff units. 

Initially, planning focused on overhauling the existing system. Once the 
comprehensive delivery system under PCTEA control was in place, planning became 
increasingly oriented toward operational matters. A great deal of effort has 
been directed at refining and expanding on a basic design that is considered 
to be sound. This refining effort is aided by extensive formal and J-nformal 
monitoring and supported by a high quality MIS. With the development of an 
impressive evaluation capacity, planning now includes a strategic component. 
Evaluation studies are being undertaken that will test basic assumptions about 
the effectiveness of different employment and training strategies. 

The PCTEA is committed to serving the disadvantaged in CETA programs, and 
their record of participant service indicates that this goal has been achieved. 
They also believe that disadvantaged clients often require more than one pro- 
grammatic service on their road to job readiness, and the delivery system 
has been designed to meet this need. This approach has led to relatively high 
costs per placement, and lower placement rates (calculated on the basis of total 
enrollment) than might be expected. Nevertheless, the PCTEA has generally achieved 
placement rates that are higher than the national average, which in combination 
with their low rates of non-positive termination suggests that the program is 
achieving its basic goals. 



SAN FRANCISCO, CALIFORNIA 

The San Francisco prime sponsorship encompasses the city and county of 
San Francisco, a consolidated, general purpose political unit. Geographically 
small, the city is a densely populated urban area with an unemployment rate 
that is consistently higher than the national average. 

One of five counties in the San Francisco-Oakland Bay Area SMSC, San 
Francisco has a white collar job base with relatively few industrial and 
manufacturing jobs, reflecting the predominance of commerce, insurance, 
finance, and government business in the city. Most job openings and 
replacement needs for which CETA participants can compete are in the 
secretarial and clerical areas* 

The polyglot population in San Francisco consists of a large number of 
identifiable ethnic, racial, and cultural groups. The population's diversity 
is reflected in the choice of significant segments to be served in CETA 
programs. This diversity also helps to account for the large number of 
subcontractors funded to deliver manpower services there were more than 30 
in FY 78 for Title I alone. 

Subcontractors are chosen every year througih a formal RFP. Selection 
decisions and funding level decisions are made by the Manpower Planning Council 
(MFC), in close interaction with the planning staff. All service deliverer 
contracts are performance-based and contain quantitative and qualitative 
goals. Refunding of service deliverers depends largely on past performance. 
Service deliverers are closely monitored by the operations staff. 

The manpower delivery system in San Francisco allocates nearly 75% of 
the Title I expenditures to occupational training, vocational ESL, and OJT 
in order to make CETA participants employable and competitive in the labor 
market . 

The public planning process entails a close working relationship between 
the planning staff and members of the MFC. The MFC is an active and aggressive 
council and has a very important role in planning decisions. The MFC 
determines how money will be allocated, who will be served, and which 
subcontractors will be funded. It also evaluates program performance each 
year. The information from the prime sponsor's MIS unit is an essential 
component of all of the planning decisions made by the MFC and the planning 
staff. 

Program performance in San Francisco is good both in quantitative and 
qualitative terms. The prime sponsorship's performance has typically been 
better than national averages on placement rate, non-positive termination 
rate, and cost per placement. The continuous attention of the staff and 
the MPC to monitoring and evaluating the performance of the system as a 
whole and of all of its individual subcontractors helps to account for the 
good performance, which has been improving over time. 



fKACUSE, NEW YOHK 

Syracuse, a single-city prime sponsorship with a declining population esti- 
ited at 177,000 in 1978 (down from 197,000 in 1970), had an unemployment rate of 
'tween 10% and 11% in the first half of 1976. The rate now hovers slightly 
ove 7%. The city's labor force is declining in size from a 1975 peak of 
i,000. The majority of its members are white (86%), male (61%), and between 
1 and 44 (51%). Blacks represent about 10% of the city's population and about 
1% of the unemployed. Manufacturing jobs are disappearing. Blue collar jobs 
e projected to comprise only 19% of the openings occurring between 1974-85, while 
Lite collar jobs dominate with 65%, and service occupations provide 16%. 

The CETA programs are all operated by the city's Office of Federal and 
:ate Aid Coordination (OFSAC) which has contracted intake, classroom training, 
id basic education to community based organizations and ES, the Skill Center, 
id the state university system respectively; and staffed assessment, counseling, 
ult and youth work experience, OJT, individual referral, job development, and 
.acement with city staff. There is also a planning staff of about 5 people 
:or Title I) and fiscal and evaluation units. They are generally competent, 
tdicated, young, and often cited as the primary reason for the program's success, 
w of them came from categorical programs . 

The program is funded at about $1.6 million annually for Title I. It 
pically allocates slightly more than 40% of this to classroom training, but 
.is 40% is being shifted to use in individual referral, with the Skill Center 
.e to make the conversion complete in FY 1979. The remaining 60% is divided 
long OJT, work experience, and client services. OJT has been deliberately 
.creased and is due to become 25% of the whole in FY 1979, at the expense of 
rk experience. 

Planning, which is central to the program, is dominated by an activist 
aff and defined in broadest terms as the construction of goals, the review 

present capabilities, and the determination of appropriate courses of action, 
ng term goals are defined, translated into short term goals, and implemented, 
nitoring, which is thorough and systematic, provides important data input, 
cio-economic data is also used, but with caution. The advisory council is 
ssive, has no important role in the selection of service deliverers, and is 
st influential in its committees, which review and check target groups, target 
cupations, service deliverer evaluations, and to a lesser degree program 
x to catch errors or suggest improvements in staff recommendations. These, 
wever, because of demonstrated staff competence, are usually trusted. 

OFSAC views CETA as one element in a centralized grants management system 
at should integrate help from different funding sources in addressing local 
eds. Staff who plan CETA are also active in planning Revenue Sharing, Public 
rks, and other federal programs. Goals for Title I include getting good marks 

standard efficiency measures (where the non-positive termination rate is 
pecially favorable), providing comprehensive and adequate individualized 
rvices without creaming, and contributing to increased public sector 
volvement and economic development activities. Progress is being made toward 
1 of these. 



Syracus-e staff members have been active in the Employment and Training 
Council of the Conference of Mayors* (The Mayor of Syracuse was President of 
the Conference .) Their activity put them in contact with progressive prime 
sponsors, increased their confidence, and kept them aware of issues. Decisions 
on the shape of Title I are made with awareness of the options and because they 
contribute to keeping Syracuse a "good news" program, but not just for 
expediecy. The program appears to be a vital and growing one that is using 
maturing manpower skills to depart decisively from categorical inheritances. 
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